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We  consider  two  queueing  control  problems  that  are  stochastic  versions  of  the  eco- 
nomic lot  scheduling  problem:  a  single  server  processes  A^  customer  classes,  and  com- 
pleted units  enter  a  finished  goods  inventory  that  services  exogenous  customer  demand. 
Unsatisfied  demand  is  backordered,  and  each  class  has  its  own  general  service  time  dis- 
tribution, renewal  demand  process,  and  holding  and  backordering  cost  rates.  In  the  first 
problem,  a  setup  cost  is  incurred  when  the  server  switches  class,  and  the  objective  is  to 
minimize  the  long  run  e.xpected  average  costs  of  holding  and  backordering  inventory  and 
incurring  setups.  The  setup  cost  is  replaced  by  a  setup  time  in  the  second  problem,  where 
the  objective  is  to  minimize  average  holding  and  backordering  costs.  In  both  problems  we 
restrict  ourselves  to  a  class  of  dynamic  cyclic  policies,  where  idle  periods  and  lot  sizes  are 
state-dependent,  but  the  N  classes  must  be  served  in  a  fixed  sequence.  Under  standard 
heavy  traffic  conditions,  these  scheduling  problems  are  approximated  by  diffusion  con- 
trol problems.  The  approximating  setup  cost  ])roblem  is  solved  exactly,  and  the  optimal 
dynamic  lot  sizing  policy  is  found  in  closed  form.  Structural  results  and  an  algorithmic 
procedure  are  derived  for  the  setup  time  problem.  A  computational  study  is  undertaken 
to  compare  the  proposed  policy  and  several  straw  policies  to  the  numerically  computed 
optimal  policy. 
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We  consider  a  queueing  system  scheduling  problem  that  is  motivated  by  a  situation 
commonly  found  in  make-to-stock  manufacturing.  The  system  consists  of  a  single  server, 
or  machine,  and  multiple  customer  classes,  which  will  be  referred  to  as  products.  Each 
product  has  its  own  general  service  time  distribution,  and  completed  units  are  placed  into 
a  finished  goods  inventory;  we  assume  that  an  ample  amount  of  (costless)  raw  material 
inventory  is  available.  Each  product  has  its  own  renewal  demand  process  that  depletes 
the  inventory,  and  unsatisfied  demand  is  backordered. 

We  analyze  two  variants  of  the  scheduling  problem.  In  the  setup  cost  problem,  a  cost 
is  incurred  when  the  machine  switches  production  from  one  product  to  another;  in  the 
setup  time  problem,  a  random  setup  time  is  incurred  when  the  server  switches  product. 
We  restrict  ourselves  to  the  class  of  dynamic  cyclic  policies,  where  each  product  is  serviced 
once  per  cycle  and  the  order  of  production  does  not  change.  Thus,  the  server  has  three 
scheduling  options  at  each  point  in  time:  Produce  a  unit  of  the  product  that  is  currently 
set  up.  change  over  to  the  next  product  in  the  cycle  (and  initiate  service  in  the  setup  cost 
problem),  or  remain  idle.  Each  product  has  its  own  costs  per  unit  time  for  holding  and 
backordering  a  unit  in  inventory.  The  objective  in  the  setup  time  problem  is  to  minimize 
the  long  run  expected  average  inventory  costs  (that  is,  holding  and  backorder  costs);  the 
objective  in  the  setup  cost  problem  is  to  minimize  the  average  inventory  and  setup  costs. 

These  problems  are  prevalent  in  many  industries  because  facilities  that  operate  in  a 
make-to-stock  mode  typically  produce  standardized  products  that  require  setups.  The 
setup  time  problem  is  more  realistic  than  the  setup  cost  problem  in  most  situations,  l)ut 
is  also  less  amenable  to  analysis.  The  setup  cost  problem,  however,  is  relevant  for  manu- 
facturing systems  that  have  embraced  just-in-time  (JIT)  manufacturing  and  internalized 
their  setup  times;  that  is,  they  incur  significant  material,  labor  and/or  capital  costs  to 
achieve  essentially  instantaneous  switchovers. 

This  dynamic  scheduling,  or  lot-sizing.,  problem  is  a  stochastic  version  of  the  classic 
economic  lot  scheduling  problem  (ELSP),  which  is  NP-hard  (Hsu  1983)  and  has  not  been 
solved  in  general.  Despite  the  vast  literature  devoted  to  the  ELSP  (see  the  survey  paper 
by  Elmaghraby  1978,  and  Zipkin  1991  for  a  list  of  more  recent  references),  its  determ- 
inistic viewpoint  has  probably  prevented  its  widespread  industrial  use:   The  solution  to 


a  deterministic  problem  in  a  make-to-stock  setting  will  not  hedge  against  uncertainty 
in  future  service  times  (e.g.,  machine  failures)  and  demand,  resulting  in  many  costly 
backorders  (see  the  numerical  results  in  Federgruen  and  Katalan  1993). 

Not  surprisingly,  the  stochastic  version  of  the  ELSP  appears  to  be  analytically  in- 
tractable. When  the  state  space  is  taken  to  be  discrete,  the  stochastic  ELSP  (or  SELSP) 
can  be  viewed  as  a  make-to-stock  version  of  the  dynamic  scheduling  problem  for  a  polling 
system,  which  is  a  traditional  (i.e.,  make-to-order)  multiclass  queue  with  setups.  In  fact, 
our  paper  can  be  viewed  as  a  companion  to  Reiman  and  Wein  (1994).  who  analyze  the  dy- 
namic scheduling  problem  for  a  two-class  polling  system.  The  SELSP  is  more  challenging 
than  the  polling  scheduling  problem,  which  also  appears  to  defy  exact  analysis,  because  of 
the  nonlinear  cost  structure  and  the  lack  of  an  imposed  boundary  at  the  origin.  Despite 
its  difficulty,  this  problem  has  been  the  subject  of  a  recent  flurry  of  activity.  Graves  (1980) 
develops  a  Markov  decision  model  for  a  one-product  problem,  and  uses  it  to  develop  a 
heuristic  for  the  SELSP  in  a  periodic  review  setting.  Leachman  and  Gascon  (1988),  Gal- 
lego  (1990)  and  Bourland  and  Yano  (1995)  develop  heuristic  lot-sizing  algorithms  for  the 
ELSP  with  stochastic  demands  that  are  rooted  in  the  solution  to  the  deterministic  ELSP; 
the  first  of  these  papers  considers  a  discrete  time  problem  with  nonstationary  demand. 
Sharifnia,  Caramanis  and  Gershwin  (1991)  employ  a  hierarchical  approach  to  develop 
heuristic  policies  for  a  stochastic  fluid  version  of  the  problem.  Federgruen  and  Katalan 
(1993,  1994)  develop  accurate  distributional  approximations  for  polling  systems,  and  use 
these  to  analyze  the  performance  of  a  class  of  periodic  base  stock  policies  for  the  SELSP. 
Anupindi  and  Tayur  (1994)  also  consider  a  class  of  periodic  base  stock  policies,  and  use 
a  simulation  based  approach  (infinitesimal  perturbation  analysis  and  gradient  search)  to 
obtain  good  base  stock  policies  for  a  variety  of  performance  measures.  Sox  and  Muckstadt 
(1995)  formulate  the  SELSP  as  a  stochastic  program  and  propose  a  heuristic  decomposi- 
tion algorithm  to  solve  it.  Qiu  and  Loulou  ( 1995)  formulate  the  problem  as  a  semi-Markov 
decision  process,  and  numerically  compute  the  optimal  solution  in  the  two-product  case; 
this  is  the  only  paper  to  date  to  gain  any  insight  into  the  nature  of  the  optimal  solution 
to  the  SELSP. 

As  in  Reiman  and  Wein,  we  employ  heavy  traffic  approximations  in  an  attempt  to 


make  further  progress  with  this  problem.  This  approach  assumes  that  the  server  must  be 
busy  the  great  majority  of  time  in  order  to  meet  average  demand  over  the  long  term.  We 
draw  upon  the  results  of  Coffman,  Puhalskii  and  Reiman  (1995a,  1995b)  to  approximate 
the  setup  cost  and  setup  time  problems  by  diffusion  control  problems.  Their  results 
allow  the  analysis  of  the  problems  to  separate  onto  two  different  time  scales.  On  the 
time  scale  over  which  the  total  workload  content  of  the  inventory  varies,  we  solve  a  one- 
dimensional  diffusion  control  problem  to  determine  the  machine  idling  policy;  this  time 
scale  corresponds  to  the  standard  heavy  traffic  normalization,  where  time  and  space  are 
compressed  by  the  factors  n  and  y/n,  respectively.  Starting  with  this  normalization,  we 
expand  time  by  a  factor  of  y/n,  so  that  both  time  and  space  are  compressed  by  \/n;  on  this 
time  scale,  the  total  workload  content  of  the  inventory  is  fixed  and  the  individual  inventory 
levels  vary  according  to  a  fluid  model.  Here,  deterministic  optimization  is  used  to  derive 
the  state-dependent  lot  sizes  when  the  server  is  busy.  These  two  controls  are  decoupled 
in  the  setup  cost  problem,  and  a  closed  form  solution  is  derived.  In  the  setup  time 
problem,  however,  the  lot  sizes  derived  under  the  fluid  scaling  impact  the  drift  of  the  one- 
dimensional  diffusion  process  that  arises  under  the  standard  heavy  traffic  normalization. 
In  this  case,  we  develop  some  structural  results  about  the  optimal  solution  and  provide  an 
algorithmic  procedure  to  solve  the  one-dimensional  diffusion  control  problem.  Although 
no  weak  convergence  results  are  provided  to  rigorously  justify  our  analysis,  we  have 
no  doubt  that  our  results  are  essentially  correct;  consequently,  we  conjecture  that  our 
proposed  policies  are  asymptotically  (in  the  heavy  traffic  limit)  optimal  for  the  two- 
product  problems,  where  the  optimal  policies  are  dynamic  cyclic  policies. 

In  an  attempt  to  both  assess  the  effectiveness  of  some  simpler  policies  and  synthes- 
ize some  of  the  existing  literature  on  the  SELSP,  we  perform  a  heavy  traffic  analysis  of 
two  straw  policies  that  are  closely  related  to  those  considered  by  Federgruen  and  Katalan 
(1993)  and  by  Sharifnia,  Caramanis  and  Gershwin.  Then  a  computational  study  is  under- 
taken that  compares  our  proposed  policies  and  these  two  straw  policies  to  the  numerically 
derived  optimal  policy  for  a  variety  of  two-product  problems;  several  five-product  prob- 
lems are  also  examined. 

The  explicitness  of  our  results  reveals  a  number  of  new  and  unexpected  insights  into 


the  nature  of  the  optimal  solution  to  the  SELSP.  Readers  who  are  not  curious  about  the 
mathematical  details  but  who  wish  to  obtain  a  deeper  understanding  of  the  SELSP  may 
find  it  useful  to  bypass  the  heavy  traffic  analysis  and  focus  on  §1.8,  §2.5  and  §3.4,  where 
the  key  insights  and  observations  are  collected. 

The  remainder  of  the  paper  is  divided  into  four  sections.  §1  and  §2  are  devoted  to 
the  analysis  of  the  setup  cost  and  setup  time  problems,  respectively.  The  computational 
study  is  described  in  §3  and  concluding  remarks  are  made  in  §4. 

1.    THE  SETUP  COST  PROBLEM 

1.1.  Problem  Description.  A  single  server,  or  machine,  produces  A'^  types  of 
products.  Each  product  z  =  1.2  . . .  ,  N  has  its  own  general  service  time  distribution  with 
service  rate  n,  and  coefficient  of  variation  (standard  deviation  divided  by  the  mean)  c^,. 
The  demand  for  each  product  follows  a  renewal  process,  where  the  mean  and  coefficient 
of  variation  of  the  interdemand  times  are  given  by  A~^  and  cj,,  respectively.  Our  results 
easily  generalize  to  correlated  compound  renewal  processes;  see  Reiman  (1984)  for  details. 
Hence,  the  traffic  intensity,  or  long  run  average  server  utilization,  is  p  =  Yl',=i{^i/^'t)  a'ld 
pt  —  ^i/Pi  is  the  utilization  for  product  i. 

Let  I,(t)  denote  the  number  of  units  of  product  i  in  inventory  at  time  t.  A  service 
completion  of  product  i  at  time  t  increases  [,{t)  by  one,  and  a  unit  of  product  i  demanded 
at  time  t  depletes  It(t)  by  one.  If  a  demand  for  a  unit  of  product  i  occurs  when  /,(/)  <  0. 
we  say  that  this  unit  of  demand  is  backordered. 

Since  the  scheduler  follows  a  dynamic  cyclic  policy,  only  three  options  are  available 
at  each  point  in  time:  Produce  the  product  that  is  currently  set  up,  switch  over  and 
initiate  production  of  the  next  product  in  the  cycle,  or  sit  idle.  Because  a  setup  is  costly 
and  instantaneous,  the  option  of  switching  to  another  product  and  then  idling  is  clearly 
suboptimal  and  will  not  be  considered.  The  server  is  assumed  to  work  in  a  preemptive- 
resume  manner,  although  the  subsequent  heavy  traffic  analysis  is  too  crude  to  distinguish 
between  this  and  the  non-preemptive  discipline. 

A  cost  h,  is  incurred  per  unit  time  for  holding  a  unit  of  product  i  in  inventory,  and  a 
cost  b,  is  incurred  per  unit  time  for  backordering  a  unit  of  product  i.  Let  us  also  define  the 


cost  indices  /?,  =  ^,7^,  and  b,  =  6,/^f,,  which  represent  holding  and  backorder  cost  rates  per 
unit  of  expected  work  in  inventory.  These  indices  play  a  key  role  in  our  analysis,  and  are 
the  analog  to  the  classic  "c/^i"  index  in  stochastic  scheduling  theory  (e.g.,  Cox  and  Smith 
1961).  Without  loss  of  generality,  the  products  are  numbered  so  that  hj\!  =  mini<,<,v  h,. 
For  notational  convenience,  we  assume  that  6/v  =  mini<,<Arft!,  so  that  the  product  with 
the  smallest  holding  cost  index  also  has  the  smallest  backorder  cost  index;  we  show  later 
that  this  restriction  does  not  change  the  analysis  from  that  of  the  more  general  case. 
Consequently,  product  A^  will  often  be  referred  to  as  the  least  cost  product. 

Because  products  are  produced  once  per  cycle  in  a  fixed  sequence,  the  performance 
of  the  system  depends  on  the  setup  costs  only  through  the  total  setup  cost  per  cycle, 
a  quantity  we  denote  by  A'.  Of  course,  the  optimal  sequence  of  products  can  be  found 
by  solving  a  travelling  salesman  problem,  where  the  "distance"  between  "cities"  i  and  j 
is  given  by  the  setup  cost  incurred  when  switching  from  product  i  to  j.  For  notational 
convenience,  we  simply  assume  that  a  setup  cost  K/N  is  imposed  whenever  the  server 
switches  from  one  product  to  another. 

The  scheduling  policy  determines  the  inventory  process  /,  and  the  counting  process 
J,  where  J{t)  denotes  the  cumulative  number  of  setups  incurred  by  the  server  up  to  time 
t.  Although  the  scheduling  policy,  and  hence  the  stochastic  processes  /,  and  J,  can  be 
rigorously  defined  in  terms  of  a  sequence  of  starting  and  stopping  times  for  each  product, 
we  omit  this  detailed  specification  because  it  is  not  required  in  our  subsequent  analysis. 
Our  problem  is  to  find  a  scheduling  policy,  which  is  nonanticipating  with  respect  to  /;,  to 
minimize 
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where  x'^  =  max(.r,0)  and  ,r~  =  min(.r,0). 

1.2.  The  Heavy  Traffic  Normalizations.  The  heavy  traffic  averaging  principle 
described  in  §1.3  allows  us  to  compactly  characterize  the  class  of  dynamic  cyclic  policies 
in  heavy  traffic.  Since  we  will  be  optimizing  over  this  class  of  policies,  there  is  no  need 
to  precisely  formulate  the  (more  general)  control  problem  that  approximates  (1)  in  heavy 
traffic,  and  it  will  suffice  to  describe  the  heavy  traffic  conditions  and  normalizations. 


\imsup  ^E\  r  J2ChJ+(t)  +  bj-it))dt  +  ^J{T)i  (1) 


The  standard  heavy  traffic  condition  states  that  the  server  must  be  busy  the  majority 
of  time  to  meet  average  demand.  To  prove  a  heavy  traffic  limit  theorem,  one  typically 
defines  a  sequence  of  control  problems  indexed  by  ??,  where  the  traffic  intensity  of  the  n' 
system  approaches  unity.  Because  a  limit  theorem  will  not  be  proved  here,  we  ease  the 
notational  burden  by  assuming  the  existence  of  a  large  integer  n  satisfying  \/n\l  —  p)  =  c, 
where  c  is  positive  and  0(1).  Our  recommended  policy  will  turn  out  to  be  independent 
of  the  heavy  traffic  scaling  parameter  /).  Let  the  process  W]  denote  the  workload  content 
embedded  in  the  finished  goods  inventory  /,.  If  /,(/)  is  positive  then  W,{t)  represents 
the  sum  of  the  service  times  for  units  in  inventory,  and  if  /,(/)  is  negative  then  W,(t) 
represents  the  amount  of  time  that  a  continuously  busy  server  requires  to  clear  product 
Vs  backlog.  The  process  W^  will  be  referred  to  as  the  workload  process  for  product  i.  The 
standard  heavy  traffic  scalings  will  be  employed  to  define  the  normalized  inventory  process 
I,(t)  —  I,{ni)/y/n  and  the  normalized  workload  process  W,{t)  =  W^int)/ \/ri  (throughout 
the  remainder  of  the  paper,  a  "tilde"  denotes  the  unsealed  version  of  the  normalized 
process).  Although  the  workload  process  H',  is  not  observable  by  the  scheduler  when 
inventory  is  backordered,  the  normalized  workload  process  W^  is  more  convenient  for 
analysis  than  the  normalized  inventory  process  1^.  However,  the  linear  identity  I,  =  i^iiW,, 
which  is  known  to  hold  for  a  wide  variety  of  queueing  systems  in  heavy  traffic,  will  be 
employed  for  translating  the  solution  of  the  heavy  traffic  control  problem  into  a  scheduling 
policy  that  is  expressed  in  terms  of  the  unsealed  inventory  process  (/i, . . . ,  /,v)- 

The  objective  function  (1)  consists  of  inventory  costs  and  setup  costs.  To  perform 
a  nontri\  ial  heavy  traffic  analysis,  we  need  these  two  costs  to  be  of  the  same  order  of 
magnitude  in  the  approximating  control  problem.  Since  the  inventory  contents  and  time 
are  normalized  in  the  heavy  traffic  control  problem,  the  relative  magnitude  of  the  inventory 
costs  and  setup  costs  also  need  to  be  scaled  in  order  to  achieve  the  aforementioned  balance. 
The  appropriate  normalization  (see  Reiman  and  Wein  for  details)  is  to  reduce  the  setup 
cost  A  by  a  factor  of  n  relative  to  the  inventory  costs,  and  consequently  we  leave  the 
inventory  costs  h,  and  6,  unsealed  at  0(1),  and  define  the  normalized  setup  cost  k  =  h'/n. 
That  is.  if  the  setup  cost  is  smaller  than  0{n),  then  setup  costs  will  be  negligible  in  heavy 
traffic,  and  if  they  are  larger  than  0(»),  then  they  will  dominate  the  inventory  costs  in 
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heavy  traffic.  Thus,  heavy  traffic  conditions  for  the  setup  cost  problem  require  the  server 
to  be  busy  most  of  the  time,  and  require  the  setup  cost  to  be  very  large. 

1.3.  A  Preliminary  Heavy  Traffic  Result.  A  recent  heavy  traffic  result  obtained 
by  Coffman,  Puhalskii  and  Reiman  (1995a)  provides  the  basis  for  our  analysis.  This 
result,  which  will  be  referred  to  as  the  CPR  result,  considers  a  traditional  multiclass  (the 
result  is  only  proved  for  the  A^  =  2  case,  and  is  conjectured  for  A'^  >  2),  single  server 
queueing  system  with  renewal  arrival  processes,  general  service  time  distributions  and 
no  setup  times.  A  cyclic  exhaustive  polling  mechanism  is  employed:  The  server  serves 
each  class  to  exhaustion  and  then  switches  to  the  next  class.  Let  us  reuse  the  service  time 
notation  /u,  and  Cj,,  and  allow  the  demand  parameters  A,  and  Cd,  in  the  SELSP  to  also 
characterize  the  arrival  process  to  this  queueing  system.  If  we  let  Vj  denote  the  normalized 
workload  for  class  /'  in  the  queue,  then  limit  theorems  in  Iglehart  and  Whitt  (1970)  and 
Reiman  (1988)  imply  that  the  total  workload  process  V  =  Ylv=i  K  is  well  approximated 
under  heavy  traffic  conditions  by  a  reflected  Brownian  motion  on  [0,oo)  with  drift  — c 
and  variance 

-^  =  £^(4 +  4)-  (2) 

See  Harrison  (1985)  for  a  definition  of  this  process,  which  we  denote  by  RBM  {—c,a^). 

The  CPR  result  allows  us  to  obtain  information  about  the  multidimensional  workload 
process.  It  provides  a  time  scale  decomposition  for  this  system:  On  the  time  scale  giving 
rise  to  RBM  for  the  total  workload,  the  N— dimensional  workload  (V'i,...,Vyv)  process 
moves  infinitely  quickly  (in  the  asymptotic  limit)  along  a  constant  workload,  piecewise- 
linear  path  connecting  points  where  the  server  exhausts  a  product.  In  the  two-product 
case,  the  path  is  the  line  segment  from  (O.V)  to  (V, 0);  in  the  case  of  three  identical 
products  (with  the  same  service  and  demand  rates),  the  path  consists  of  the  line  segments 
connecting  the  points  (0,  V'/S,  2V/3),  (V73,2V73,0)  and  (2V73,  0,  V/S). 

We  make  the  crucial  assumption  that  the  time  scale  decomposition  holds  not  just  for 
the  cyclic  exhaustive  policy  (which  corresponds  to  a  cyclic  base  stock  policy  in  the  SELSP 
problem),  but  for  more  general  cyclic  policies;  this  assumption  should  not  raise  any 
objections  because  the  time  scale  decomposition  is  inherent  in  the  heavy  traffic  scaling. 


The  time  scale  decomposition  gives  rise  to  an  averaging  principle  that  facilitates  our 
analysis  in  the  following  way:  If  we  consider  the  heavy  traffic  normalization  and  then 
expand  the  time  scale  (i.e.,  slow  down  time)  by  a  factor  of  x/n,  then  the  total  workload 
stays  constant  and  the  A^-dimensional  workload  behaves  as  a  fluid  (i.e.,  moves  at  a  finite 
rate  in  a  deterministic  fashion). 

1.4.  An  Overview  of  the  Analysis.  It  is  useful  to  view  the  control  policy 
as  consisting  of  two  interrelated  decisions:  A  busy/idle  policy  and  a  dynamic  lot-sizing 
policy  that  specifies  what  the  server  should  do  while  working.  We  begin  by  characterizing 
the  busy/idle  policy.  The  CPR  result  and  the  well  known  relationship  between  queueing 
systems  and  production/inventory  systems  (e.g.,  Morse  1958)  imply  that  the  system  state 
of  the  heavy  traffic  control  problem  is  the  one-dimensional  total  workload  process  W  = 
Xli=i  11'm  which  measures  the  total  machine  time  embodied  in  the  current  finished  goods 
inventory.  Furthermore,  since  setup  times  are  zero,  the  total  workload  process  is  only 
affected  by  the  server's  busy/idle  policy,  not  by  how  often  the  server  switches  among 
products.  Hence,  the  only  reasonable  form  of  the  optimal  busy/idle  policy  is  for  the  server 
to  stay  busy  if  \V(t)  <  wq  and  to  idle  if  W{t)  >  uiq,  for  the  unspecified  control  parameter 
u'o.  The  quantity  Wq  will  often  be  referred  to  as  the  idling  threshold,  and  can  be  viewed 
as  an  aggregate  base  stocA'  level.  The  CPR  result  and  the  one-to-one  relationship  between 
queueing  systems  and  production/inventory  systems  (i.e.,  W{t)  =  wq  —  V'(/))  imply  that 
the  total  workload  process  W  is  a  RBM  (c,  cr^)  on  (—00.  Wq]  under  this  busy/idle  policy. 

■Although  the  lot-sizing  policy  does  not  influence  the  total  workload  W .  it  does  affect 
the  rate  at  which  inventory  costs  and  setup  costs  are  incurred  when  W(t)  =  w.  Therefore, 
a  two-step  procedure,  with  each  step  being  performed  at  a  different  time  scale,  can  be 
used  to  find  the  optimal  dynamic  cyclic  policy.  In  the  first  step,  CPR's  averaging  principle 
is  used  to  find  the  lot-sizing  policy  that  minimizes  the  average  (inventory  plus  setup) 
cost  incurred  as  the  individual  inventory  levels  oscillate  deterministically  while  the  total 
workload  remains  constant  at  w.  let  us  call  the  resulting  minimum  cost  c{w).  By  solving 
a  family  of  deterministic  optimization  problems  indexed  by  the  total  workload  iv,  we  are 
able  to  construct  a  dynamic  (i.e.,  workload-dependent)  lot-sizing  policy.    In  the  second 


step,  we  find  the  aggregate  base  stock  level  ivq  that  minimizes  the  long  run  average  cost 


/- 


c(ti.) 4e-''< ""'-"'>/''  f/u),  (3) 


oo  (7^ 


where  we  have  used  the  fact  (e.g.,  Harrison)  that  RBM  {—c,a^)  on  [0,oo)  has  an  expo- 
nential stationary  distribution  with  parameter  '2\c\/a^. 

In  §1.5,  we  calculate  the  average  cost  incurred  by  a  generic  dynamic  cyclic  policy 
when  the  total  workload  equals  w.  The  optimal  heavy  traffic  cyclic  policy  is  found  in 
§1.6.  Finally,  the  heavy  traffic  normalizations  are  reversed  in  §1.7  to  obtain  the  proposed 
scheduling  policy  as  a  function  of  the  unsealed  inventory  levels  and  the  server  location. 

1.5.  Construction  of  Dynamic  Cyclic  Policies.  The  goal  of  this  subsection 
is  to  find  the  cost  associated  with  any  dynamic  cyclic  policy  when  the  total  workload 
\'V(t)  =  w.  The  CPR  result  makes  this  calculation  tractable:  Starting  with  the  heavy 
traffic  normalization  Wi{t)  =  Wt{nt)/s/n,.  we  slow  down  time  by  a  factor  of  y/n  to  obtain  a 
fluid  scaling,  W,{t)  =  Wi{y/nt)/ ,/n.  At  this  time  scaling,  the  process  W{t)  =  \V(nt)/y/n 
is  fixed  at  the  value  u\  and  the  N—  dimensional  workload  W,  moves  at  a  finite  rate  in 
a  deterministic  manner.  Because  many  fluid  cycles  occur  before  the  diffusion  process  W 
changes  value,  it  suffices  to  analyze  the  deterministic  behavior  of  one  cycle. 

A  cyclic  policy  is  essentially  characterized  by  the  lot  size  for  each  product  (or  ecjui- 
valently,  since  the  analysis  is  deterministic,  the  length  of  time  each  product  is  served  in  a 
cycle).  Because  idleness  is  only  incurred  when  the  total  workload  reaches  a  certain  base 
stock  level,  we  assume  that  no  idleness  is  incurred  during  a  cycle.  When  W{t)  =  il\  a 
cyclic  policy  is  best  viewed  as  a  closed  A^  — dimensional  deterministic  path  in  the  constant 
workload  hyperplane  J2i=i  '^'-'i  =  ^'-''■^  that  is,  the  process  traverses  the  same  path  repeatedly, 
once  per  cycle. 

Although  a  cyclic  policy  can  be  specified  in  many  ways,  we  choose  a  particular  char- 
acterization that  is  convenient  for  analysis.  A  cyclic  policy  (or,  equivalently,  the  closed 
loop  generated  by  the  policy)  will  be  defined  by  A^  -|-  1  quantities:  The  cycle  length  t 
and  the  cycle  center  x^  =  ( jj, .  .  . ,  x%).  These  control  parameters  are  actually  functions 
of  the  total  workload  w,  but  this  dependence  will  be  suppressed  for  improved  readability. 


time 


Figure  1:  Workload  fluctuation  over  a  cycle. 

The  cycle  length  r  is  the  length  of  time  recjuired  to  perform  a  cycle,  and  .r^  is  product 
Vs  "center  of  fluctuation",  or  equivalently.  the  average  amount  of  this  product's  invent- 
ory over  the  course  of  a  cycle.  Because  the  transient  effects  associated  with  initiating  or 
temporarily  moving  a  cycle  vanish  in  the  heavy  traffic  time  scaling,  the  cycle  center  .r'" 
can  be  placed  anywhere  in  the  constant  workload  hyperplane. 

We  begin  by  examining  the  deterministic  behavior  of  the  individual  product  workload 
levels  W,  under  a  cyclic  policy  when  \\'{i)  =  w.  For  the  system  to  remain  balanced,  the 
amount  of  each  product  produced  per  cycle  must  equal  the  amount  demanded,  and  hence 
each  product  must  be  produced  a  fraction  p,  of  the  time;  we  assume  that  p  equals  one 
throughout  this  fluid  analysis,  so  that  the  server  is  busy  throughout  the  cycle.  Thus,  for 
an  arbitrary  instantaneous  total  workload  w  and  cycle  time  r,  each  product  i  must  be 
serviced  for  p,T  units  of  time  per  cycle.  Therefore,  when  the  machine  is  servicing  product 
i,  the  work  content  in  this  product's  inventory  is  depleted  at  rate  p,  and  is  replenished 
at  rate  one,  and  W,  increases  at  the  fixed  rate  1  —  p,  for  p,T  units  of  time  per  cycle.  For 
the  remaining  (1  —  p,)t  time  units  in  the  cycle  when  product  i  is  not  being  produced, 
the  workload  inventory  is  decreasing  at  rate  p,.  To  uniquely  determine  the  behavior  of 
a  cyclic  policy,  a  reference  starting  point  also  needs  to  be  specified.  We  use  .r^,  product 
?'s  average  inventory  level,  as  the  reference  point.  Readers  are  referred  to  Figure  1  for  a 
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reinforcement  of  these  notions. 

The  cost  of  a  cycHc  policy  can  be  expressed  in  terms  of  the  cycle  length  r,  the  cycle 
center  .r*^  (an  A^— dimensional  vector)  and  the  total  workload  w.  To  find  the  average 
inventory  cost  for  product  i  per  unit  time,  c,(r,  j-j',  if),  we  integrate  the  inventory  cost 
over  the  cycle  and  divide  by  the  cycle  length.  The  average  cost  breaks  down  into  three 
regions  depending  on  whether  the  product  is  entirely  held  throughout  the  cycle,  is  held 
and  backordered  or  is  entirely  backordered.  Figure  1  allows  us  to  deduce  that 


c,(r,  ,r^,  ty)  =  < 


h,x'^  if  .r^  >  iMiz^il 


(4) 


-M^  if  J^<   -'^'^'-'^'^ 


The  total  average  cost,  c{t,x'',w),  which  includes  inventory  and  setup  costs,  is 

1.6.  The  Optimal  Dynamic  Cyclic  Policy.  Our  cyclic  policy  consists  of  three 
controls:  The  aggregate  base  stock  level  wo  and,  for  each  total  workload  level  u',  the  cycle 
length  T  and  the  jV  — dimensional  cycle  center  x"'.  In  this  subsection,  the  optimal  dynamic 
cyclic  policy  is  derived  in  three  stages:  (i)  find  the  optimal  cycle  center  x"^  in  terms  of 
arbitrary  w  and  r,  (ii)  optimize  over  the  cycle  time  r  in  terms  of  an  arbitrary  u',  and 
(iii)  substitute  the  derived  cost  function  c{ui)  into  equation  (3)  and  find  the  optimal  idling 
threshold  wq. 

The  Optimal  Cycle  Center.  We  begin  by  showing  the  existence  of  a  cost-minimizing 
cycle  center  .r^  for  a  given  total  workload  w  and  cycle  length  r.  Note  that  the  cost  func- 
tion c(t,x^,w)  is  differentiable  with  respect  to  x"^  and  its  derivative  is  continuous.  If  one 
ignores  the  constant  workload  constraint  X]!I,  ^'i  =  iv,  for  fixed  r  the  cost  function  in 
terms  of  x"^  is  piecewise-quadratic  with  linear  edges;  its  second  derivative  is  a  nonnegat- 
ive  step  function.  Thus  c{t,x'^,w)  is  convex  and  the  restriction  of  the  cost  function  to 
the  constant  workload  hyperplane  determined  by  w  is  also  convex.  This  fact  implies  the 


existence  of  a  solution  to  the  constrained  minimization  problem:  Choose  x'''  to  minimize 
c(t,x'^,w)  subject  to  Yli=i  ^"l  —  w- 

Now  we  use  the  cost  function  in  (4)  to  find  the  optimal  cycle  center.  The  total 
workload  constraint  can  be  used  to  eliminate  one  variable  and  express  the  cost  function 
as  a  piecewise  polynomial  function  of  N  —  1  variables.  Any  N  —  1  components  of  x'^  can  be 
used.  Over  the  constant  workload  hyperplane,  the  polynomial  order  of  the  N  —I  variables 


fluctuates  between  one  and  two  depending  on  whether  |x^'|  >  - 


Pi(i-p,) 


—  or   x*:'    < 


Tp,{i-p, 


respectively.  For  the  gradient  to  be  equal  to  zero,  each  of  the  iV  —  1  variables  must  be 
quadratic.  Consequently,  at  the  optimal  .r*-",  at  least  .'V  —  1  of  the  Ci{t,  .rj",  u')'s  are  of  order 
two,  with  the  remaining  component  possibly  being  linear.  To  see  this,  suppose  that  some 
of  Ci(r,  .rj',  u')'s  are  not  of  order  two,  and  let  j  denote  the  index  of  such  a  term.  If  we 
eliminate  Xj,  the  gradient  equation  can  then  be  written  as 


V,c 


N  ( 

V 


1  =  1 

<*1 


\ 


T,  W 


Y.x%w 


1  =  1 


/ 


0  . 


This  equation  will  have  a  solution  only  if  the  remaining  A^  —  1  c,{t^x'j,  u')'s  are  quadratic. 
The  following  proposition  greatly  simplifies  our  analysis. 

Proposition  1    ff  there  are  only  N  —  \    quadratic  Cj(r,  .r^,u')   ter'ms  in  the  total  cost 
function  at  the  optitiial  x" ,  theu^  the  linear  term  must  be  C!\]{t,x']^,w). 


Proof:  This  fact  is  most  easily  seen  by  examining  the  function  Ct{T,  .r,",  a')+c.v(T,  .ry^,  w] 
where  Cj  is  linear  and  c,\  is  quadratic.  By  (4),  the  sum  is 


h,(x1)^  +  b,{xX  +  ikx  +  h 


Nl 


rp,v(l  -  Pn)    ,        bN  +  h 


8 


+ 


N 


2tpn(1  -  pj^ 


^2,^'V-6N„.         (6) 


+ 


'N- 


The  tradeoff  between  x^  and  .r^v  can  be  examined  by  looking  at  this  sum  along  the  line 
x'i  +  xl;  =  w'  (with  iv'  arbitrary).  Substituting  u''  — x^  for  .r^'  into  equation  (6)  and  taking 
the  derivative  with  respect  to  x%-  yields 

-h,  H ^;^ 1 rX\,     if  X^=^r  >  w\ 


2 


Tp^{l   -  p? 


'N 


■N 
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Since  the  quadratic  region  of  j^  is  restricted  to  the  region  |.r5v|  <  LpkL^ipnI^  jj  follows 
that  the  c}uantity  in  (7)  is  less  than  or  equal  to  /?Ar  —  /?,,  and  the  quantity  in  (8)  is  greater 
than  or  equal  to  6,  —  bj^.  Hence,  neither  (7)  nor  (8)  can  equal  zero  unless  /?,  equals  hj^j 
for  x'i^j  >  w'  or  b,  equals  6/v  for  j/y  <  w'.  If  the  holding  or  backorder  costs  are  equal, 
then  the  optimal  .r^  satisfies  \x%\  =  LPml^Lmi^  resulting  in  multiple  optimal  solutions 
along  the  line  .r^  +  x%  ^  w'  with  |x^|  >  '^^'^^"^^'  and  \x1\  >  lM|z£il.  Although  many 
of  these  solutions  lie  in  the  region  where  products  i  and  A^  are  linear,  one  of  the  optimal 
solutions  occurs  when  at  least  A^  —  I  of  the  cost  components  are  quadratic  (or  on  the 
border  between  quadratic  and  linear).  Hence,  if  there  is  a  linear  cost  component  at  the 
optimal  .r*^,  it  will  be  c/v(t,  t*-^,  u»).   | 

As  a  consequence,  the  optimal  cycle  center,  or  average  amount  of  inventory  per  cycle, 
for  product  ;  <  A^  is  restricted  to  the  region  [— rp,(l  —  p,)/2,Tp,(l  —  p,)/'2].  whereas 
product  A^'s  cycle  center  can  be  arbitrarily  far  from  zero.  Intuitively,  this  fact  suggests 
that  product  A^.  which  is  the  least  cost  product  by  our  indexing  convention,  is  the  product 
that  will  hold  the  excess  or  deficit  amounts  of  work  when  the  total  workload  w  fluctuates 
far  from  zero. 

We  now  use  Proposition  1  to  find  the  optimal  cycle  center  .r"-".  Without  loss  of  gen- 
erality, the  workload  constraint  is  used  to  eliminate  .r^  from  the  cost  function,  so  that 
.T^'  =  w  —  Y^tJl^  x'i.    To  find  the  optimal  center,  we  take  the  gradient  of  (5)  and  set  it 

equal  to  zero: 

N-l  N-l  7 

V,c[X;Q(r,x^u;)  +  c^(r,a>-  ;^  .r^  u.)  + -]  =  0.  (9) 

At  this  point,  we  do  not  know  whether  c/v(t,  .r^,  ly)  is  linear  or  quadratic.  Let  x"^  be 
the  (A^  —  l)-dimensional  vector  that  solves  (9)  under  the  assumption  that  all  A^  of  the 
c,(r,  J-"',  u;)'s  are  quadratic  in  jj^.  Taking  the  (A^  —  l)-dimensional  gradient,  we  find  that 
x^  satisfies  {w  and  r  multiply  their  vectors  component-wise  in  the  analysis  below) 

-^^"^  -  71  - -72  =  0,  (10) 

T  T 


where 


A 


b,+h, 


+ 


/?A'(1  -  Pn) 


(11) 


7i 


72     = 


2  2 


^N+^N 


pn{^-pn) 


;i2) 


131 


Th 


us. 


.r^  =  tA     7i  +  wA     72, 


14) 


w 


here  the  matrix  elements  of  .4      are 


On 


Pi(l-P. )  Pj(l-Pj) 
h,  +  h,        hj+h, 

Y^.v     Pi(i-P() 
^^'=1    b,+h, 


for     «  7!^  J,      and 


'15) 


Oin 


6,  +  h. 


Y^iV     Pi(i-Pi) 
^1=1     hi+hi 


;i6) 


If  I w  —  Z!!^i ^  -J"?!  ^  ^^^''   ^^^  then  C/vIt,  x^,  w)  is  indeed  quadratic  and  .rj'  determines 


the  optimal  center:  x'i  =  x'j  for  ;  <  A'  and  x'^  =  »'— Xl,-/  -'"i-  If  |ft'~I2,=/  -^'i^l  >  ~ 
we  must  solve  the  multivariate  gradient  equation  with  the  linear  form  of  c.\'(r,  ,r^ 


rpN(l-PN) 


N 


lU] 


With  this  substitution,  equation  (9)  decomposes  into  univariate  expressions  of  the  form 


2  T/9,(l   -  p,) 

h,  -  6,    ,       6,  +  h^ 


+ 


2  Tp,{\  -  p, 


x:  +  b 


N 


0    if  w  -  E 


1       ■'  !     ^ 


2 


(17) 
(18) 


Putting  the  results  from  (14)  and  (17)-(18)  together,  we  obtain  a  complete  expression  for 
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the  optimal  cycle  center.  For  i  <  N  we  have 


(       Tp,(l-p.) 
h,+h, 


+    ll 


N 


if  »'  -  E!IT'  ^^    >    ''^^2"'''^ 


(19) 


rp.jl-p,) 
b,  +  h, 


}N 


if "' -  e!It' a^?  <  -'"^^2"'''' 


where  a,  is  the  ith  row  of  .4"'.  The  last  component,  i^,  is  equal  to  10  —  J2i=i^  •^?- 

The  Optimal  Cycle  Time.  The  optimal  cycle  time  r  can  be  derived  by  simple 
calculus.  Substituting  the  optimal  cycle  center  .r^  into  (4)  yields  the  average  inventory 
cost  for  each  product  as  a  function  of  r  and  it;.  For  i  <  N  this  cost  is  given  by 


Ct{T,w)  = 


^!h_Li^T(y,  ■  7i  +  wa,  ■  72] 


Ih  J-  h  \L£A1:zpA  _u  ^P<i^-P') 
\iJ,-t  fit)         g  -t    2{b.+h,) 


4^-6 


N 


th^-b 


N 
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III 


(20) 


where  case  I  is  shorthand  for  the  region  »'  — E!Ii'  ^1  >  ""^^^2  ''^^  ■>  ^^^^  ^^  '^  l^i'~E,Ii^  -^i^l  ^ 
I^^d^fMl,  and  case  III  is  w  -  E,=7'  4  <  -''^^[-"^l   Similarly,  product  N\  inventory 


cost  function  is 


f   Mu'-rEf=T'13^[^  +  /^N 


cn{t,w)  =  < 


PN{i-PN . 


2TPfv(l-Pw) 


2/  1  ■•r^N-l    „ 

i'  (1  -  E,=l    « 


•72  J 


hfj-b^ 


N-l 


+  ^li^(u,(l  _  E;17'  a,  .  72)  -  r  E:IT'  «.  ■  71) 


-M--rEr=7'^fe^N"-6 


'/v 


II 


III 


(21) 


The  optimal  value  of  r  is  determined  by  taking  the  derivative  of  the  total  cost  function 


Y^iLi  Cifr,  w)  +  k/r  and  solving  against  zero.  If  we  define  the  constants 


e2 


2[b,  +  h,)\      2 
b,  +  h,     , 


•lp.[\-p. 


2      /?,  -6, 
o,  -71)    +  — 7-y — O'  ■  li 


+  - TI4 ?; 74, 


9 


6 


2/)Af(l  -  pat) 
y^'  (k  +  h,)p,(l-p,)   _  p^il-p^)    f  b,  -  h,    _  ^^ 


8 


2{b,  +  h,)\      2 


N-l 
^,2p,{l-Pr) 


bi  +  hi                   2   ,       ^'V  +  hN       2 
>'  •  72)'  +  T — r73, 


2p^(l  -  pn) 


(22) 

(23) 
(24) 
(25) 
(26) 


where  73  =  1  —  Y^^-i^  o,  •  72  and  7^  =  Ylf=i'  <^i  •  7i,  then  the  optimal  cycle  length  is  given 

by 

I 


T   =    <      7«ligd±  11 


III 


Considerable  insight  can  be  gleaned  from  (27).  However,  a  discussion  of  the  insights  from 
our  analysis  is  deferred  until  §1.8. 

The  Optimal  Idling  Threshold.  With  an  expression  for  r  in  hand,  we  can  refine 
ecjuation  (3),  which  determines  the  idling  threshold  Wq.  By  using  (27)  to  ec|uate  the  cycle 
lengths  at  the  borders  of  region  II  and  its  two  adjacent  regions,  the  three  regions  can  be 
expressed  in  terms  of  the  total  workload  level.  Region  I  is  characterized  by  workload 
levels  w  >  u'l.  region  II  by  u'2  <  ic  <  Wi  and  region  III  by  w  <  W2,  where  the  workload 
level  cutoffs  are 


Wi 


102 


6-ei 


Ui 


A-, 


/6  -  e.- 

Us 


k. 


(28) 
(29) 
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By  (3)  and  (5),  the  total  average  cost  is  given  by 


/: 


Y^c,(t.x%w)  +  -]  -e-^^'^'°-U:w.  (30) 


u  =  l  ^/    '^' 


Having  solved  for  r  in  terms  of  the  system  parameters  and  w,  we  can  substitute  (27)  into 
the  inventory  cost  (20)-(21)  and  into  the  setup  cost  term  k/r  in  (30)  to  express  the  total 
average  cost  solely  in  terms  of  the  total  workload  w;  we  will  not  explicitly  write  out  this 
expression  because  it  adds  little  to  our  understanding  of  the  problem. 

Depending  on  the  idling  threshold  wq.  the  integral  can  be  broken  into  three  or  fewer 
terms  based  on  the  split  of  the  total  workload  into  three  regions  as  defined  in  (28)-(29). 
Because  of  the  irregular  form  of  r  for  w  G  [»'2,u'i],  the  integral  over  region  II  has  no 
closed  form  solution.  Therefore,  we  resort  to  numerical  methods  to  determine  the  optimal 
value  of  Wo- 

Our  derivation  of  the  optimal  dynamic  cyclic  policy  in  heavy  traffic  is  now  complete. 
Recall  that  when  the  problem  was  initially  defined,  we  assumed  that  product  N  had  both 
the  smallest  holding  and  backorder  cost  indices.  The  analysis  would  remain  unchanged, 
however,  without  this  restriction.  One  only  needs  to  introduce  new  notation  (for  example, 
A'  —  1)  to  designate  the  product  with  the  lowest  backorder  cost  index,  and  use  product 
A"  —  1  in  place  of  product  N  when  the  total  workload  level  is  less  than  zero. 

1.7.  The  Proposed  Policy.  The  final  step  in  our  analysis  is  to  employ  the  optimal 
heavy  traffic  policy  derived  in  §1.6  to  develop  a  proposed  policy  for  the  original  SELSP. 
This  is  done  in  two  stages:  We  reverse  the  heavy  traffic  scalings  to  express  the  solution 
in  terms  of  the  original  problem  parameters,  and  then  interpret  the  resulting  solution. 

If  we  replace  the  normalized  quantities  w,k  and  r  by  w/\/n,h'/n  and  i/y/n,  re- 
spectively (r  undergoes  this  normalization  because  time  is  compressed  by  \/n.  in  the  fluid 


model),  then  the  optimal  cycle  length  formula  in  (27)  becomes 


r  =  <    JiAEI+K         ii 


31) 


III 


Motivated  by  (14),  we  define  x    =  f.4   ^')i  +ibA   ^72  for  fixed  total  unnormalized  workload 
W(t)  =  lb.  Then  the  unsealed  cycle  center  x'^  is,  for  t  <  N, 


(    fp,(i-p.) 


b,-h, 


+  h 


N 


TO,  ■  7i   +  U'Q,  •  72 


(32) 


.  "^m^  [^  -  ^n]     if  -  -  Ef=T^  -n  <  - 
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and  ijy   is  equal   to   tt'  —  Z]'=i'.r^.      Hence,   the  unnormalized  average  inventory  cost, 
c,{T,y,  w).  is  equal  to  r,(f,.r'',  lu).  Upon  reversal  of  scaling,  ecjuations  (28)-(29)  become 


u'l     =     \l-^—. —  A, 


U^ 


U'2       = 


/6-6,^. 


^46 


(33) 
(34) 


Therefore,  if  wq  minimizes  equation  (30),  then  wq  =  \/nwo  will  minimize  the  unnormalized 
long  run  average  cost  expression 


U' 


A'\  2(1  -  p)    _m^,a.o-w 


-e      "' 


(u.o-u-)^/ 


W. 


(35) 


Notice  that  the  heavy  traffic  parameter  n  does  not  appear  in  (31)-(35).  To  compute  Wq  in 
§3,  we  use  Maple  V  to  numerically  solve  the  first-order  optimality  conditions  associated 
with  (35). 

Our  proposed  dynamic  cyclic  policy  for  the  SELSP  must  be  expressed  in  terms  of  the 
original  {N  +  l)-dimensional  system  state,  which  is  given  by  the  current  inventory  levels 
hit) /a'(0  and  the  server  location.  There  are  many  ways  in  which  the  unnormalized 
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policy  in  (31)-(32)  and  (35)  can  be  interpreted  for  purposes  of  implementation.  Perhaps 
the  most  natural  way  to  express  a  dynamic  lot-sizing  policy  is  to  specify  a  state-dependent 
maximum  inventory  (or  "produce-up-to" )  level  for  the  product  currently  being  produced. 
By  Figure  1,  when  the  unnormalized  total  workload  level  W{t)  equals  w.  the  maximum 
workload  level  for  product  i  is  .i'j  +  p,{l  —  p,)t/2;  of  course,  this  level  is  achieved  at  the 
instant  within  a  cycle  when  the  production  of  product  i  is  finished.  Making  use  of  the 
heavy  traffic  identity  /,  =  /ijM^j,  let  us  define  the  three  unnormalized  workload  regions  in 
terms  of  the  inventory  process  and  the  thresholds  in  (33)- (34):  Region  I  is  ZlUi  /'r'^i(0  > 
Wi,  region  II  is  IZ,=iMr'/i(0  ^  [^^'-i.ii'i]  and  region  III  is  J2,=i  1^-7^ h{i)  <  W2-  Then  our 
proposed  policy  can  be  described  as  follows:  IfY2t=i  l^~^h{t)  >  0  then  let  N  refer  to  the 
product  with  the  smallest  holding  cost  index  h^;  otherwise,  let  N  denote  the  product  with 
the  smallest  value  of  b^.  The  server  should  idle  ifJ2^=i  lJ-T^h[t)  >  Wo\  otherwise,  if  set  up 
for  product  i  <  N  then  produce  this  product  as  long  as 


f  ^/^ 


\a:6,+/i.y 
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Once  p^  '/,(0  reaches  or  exceeds  this  level,  switch  to  the  next  product.    If  set  up  for 
product  N,  then  produce  this  product  while 


S^N     ,,-lf//\    I       ff^i  P;v('-P.v)        Y^N-1  p,(1-p,)  \b,-h,     ,    i 
2^,=i  A',     ''I'^  +  V^i^         2  ^t=i       b,+h,     [~2       ^  "'^ 


p,^^lN{t)  <   < 


$2  >>  2  1^1=1      "i   ■     III 

+(E!Ii/ir'^.(0)(i-E;=7'".-72) 


E^      ii~^fli\-L      n^l  PnI^-Pn)  v^iV-1    P,(l-P 

t=i Pi  ^'y-i'^y^^y     2        2^1=1    b,+h, 


pA 


b.~h, 


'iV 


) 


I 
II 

III 


(37) 


and  then  switch  to  the  next  product  when  p]^^ij^(t)  reaches  or  exceeds  this  level. 

1.8.     Insights.       Our  analysis  reveals  numerous  insights  into  the  behavior  of  the 
optimal  policy  in  heavy  traffic.     Before  reading  this  subsection,  readers  may  want  to 


digest  the  graphs  marked  "proposed"  in  Figures  3  and  4  in  §3.2,  which  depict  the  proposed 
policies  in  both  the  symmetric  (each  product  has  the  same  parameters)  and  asymmetric 
two-product  settings. 

Three  Workload  Regions.  An  essential  feature  of  the  heavy  traffic  policy  is  its 
characterization  via  three  workload  regions,  as  described  in  (28)-(29).  There  is  sufficient 
workload  in  region  I,  significant  backorders  in  region  III,  and  region  II  represents  the 
intermediate  case  where  the  total  workload  is  in  an  interval  containing  zero. 

State- Dependent  Lot  Sizes.  Because  the  time  spent  producing  product  /  within  a 
cycle  is  /9,r,  the  optimal  cycle  length  r  determines  the  optimal  lot  sizes  in  heavy  traffic 
(and  determines  the  optimal  expected  lot  sizes  for  the  SELSP).  We  can  observe  from  (27) 
that  the  optimal  lot  sizes  are  state-dependent  when  the  total  workload  is  in  region  II.  In 
contrast,  the  lot  sizes  are  constant  in  regions  I  and  III;  in  these  regions,  surplus  or  deficit 
inventory  is  unavoidable,  and  the  trade-off  between  lot  sizes  and  setup  costs  stabilize, 
thereby  generating  constant  lot  sizes.  This  observation  and  (19)  imply  that  the  cycle 
center  .r'^  remains  constant  in  regions  I  and  III,  and  gradually  shifts  between  these  two 
points  in  the  intermediate  area  of  region  II.  It  is  worth  pointing  out  that  in  nearly  all  of 
the  deterministic  ELSP  literature  (Dobson  1987  is  a  notable  exception),  the  analysis  is 
restricted  to  policies  with  constant  lot  sizes. 

Relationship  to  the  EOQ  Model.  As  in  the  economic  order  quantity  (EOQ)  model, 
the  lot  size  is  proportional  to  the  scjuare  root  of  the  setup  cost  in  regions  I  and  III.  In 
region  II,  the  setup  cost  again  appears  in  the  numerator  of  the  square  root  term. 

Inventory  is  Focused  in  the  Least  Cost  Products.  In  region  I,  excess  inventory  is 
built  up  in  the  product  with  the  smallest  h,,  which  is  a  product  that  is  inexpensive  to 
hold  (small  h,)  and  lengthy  to  process  (small  f/,).  Similarly,  in  Region  III,  excess  negative 
inventory  (i.e.,  backorders)  is  held  in  the  product  with  the  smallest  backorder  cost  index 
6,;  this  product  is  inexpensive  to  backorder  and  has  a  long  expected  processing  time. 
In  both  regions,  inventory  is  held  in  the  least  cost  product  so  as  to  reduce  the  absolute 
value  of  the  inventory  of  the  higher  (holding  in  region  I  and  backorder  in  region  III)  cost 
products.  In  this  regard,  the  dynamic  lot-sizing  policy  derived  here  is  similar  to  the  heavy 
traffic  policy  derived  for  the  corresponding  problem  without  setups  in  Wein  (1992).    In 
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that  paper,  instantaneous  switching  causes  the  inventories  of  all  the  higher  cost  products 
to  vanish  in  the  heavy  traffic  normalization.  When  setup  costs  are  introduced,  breadth 
is  added  to  the  normalized  cycle  length  and,  for  a  fixed  total  workload,  a  "corridor"  of 
possible  inventory  states  replaces  the  least  cost  axes.  In  fact,  if  we  consider  the  special 
case  A'  =  0,  then  region  I  (region  III)  corresponds  to  w  >  0(w  <  0);  in  both  regions, 
r  =  0  and  .r^  =  w,  and  the  solution  reverts  to  that  of  Wein. 

Lot  Sizes  Grow  with  Absolute  Value  of  Total  Workload.  By  (27),  we  see  that  the 
optimal  lot  size  is  smallest  when  the  total  workload  equals  zero,  and  grows  with  the 
absolute  value  of  the  workload.  When  the  total  workload  is  near  zero,  costly  backorders 
can  be  avoided  by  switching  frequently  between  products.  In  contrast,  when  the  absolute 
value  of  the  workload  is  large,  it  is  possible  to  employ  large  lot  sizes  without  adversely 
affecting  the  inventory  costs  (because  inventory  tends  to  be  held  in  the  minimum  cost 
product  in  regions  I  and  III);  in  this  case,  it  is  advantageous  to  avoid  setup  costs  and 
produce  products  in  large  batches. 

Inventory  Levels  at  Switching  Epochs.  In  heavy  traffic,  the  maximum  normalized 
workload  for  product  i  under  our  proposed  policy  is  .r^  +  /9i(l  —  p,)t/'2.  It  follows  that  for 
i  <  N,  product  i  inventory  is  negative  (i.e.,  backordered)  when  the  server  switches  into 
product  /'  and  is  positive  when  the  server  switches  out  of  product  i.  For  product  N  the 
sign  of  the  inventory  level  during  the  switching  epochs  depends  on  the  region:  In  region 
II  product  N  inventory  is  backordered  when  the  server  switches  into  product  A'^  and  is 
positive  when  the  server  switches  out  of  product  N.  In  contrast,  product  A^  inventory  is 
always  positive  in  region  I  and  is  always  negative  in  region  III. 

Cost-Symmetric  Case.  Our  results  simplify  considerably  under  the  cost-symmetric 
case,  where  h,  =  h  and  h^  =■  h  for  all  ;'  =  1,...,A'^.  This  case  will  arise,  for  example, 
if  all  products  are  relatively  indistinguishable,  except  for  their  color.  Then  each  product 
would  be  expected  to  have  the  same  service  rate  and  inventory  cost  rates,  but  different 
demand  rates,  since  some  colors  may  be  more  popular  than  others.  Then  ^i  =  ifs  =  0 
and,  by  (28)-(29),  the  workload  always  resides  in  region  II.  In  fact,  one  can  show  that  the 
workload  resides  in  only  one  region  (region  II)  if  and  only  if  the  costs  are  symmetric.  If 
we  define  the  constant  p  =  Xlfli  Pi(^  ~  Pt)-  then  when  the  server  is  busy  and  is  set  up  for 


product  I,  this  product  is  produced  as  long  as 
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(E^,  ^-'i,(t)y  ^    -IK    ^  T:l,^i7'h{t) 
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(38) 


It  is  evident  from  this  expression  that  the  cost  structure  effects  the  lot  sizes  only  through 
the  ratio  K/{b  +  h)\  not  surprisingly,  the  lot  size  is  an  increasing  function  of  this  quantity. 

2.     THE  SETUP  TIME  PROBLEM 

2.1.  Problem  Description.  In  the  setup  time  problem,  a  random  setup  time 
(rather  than  a  setup  cost)  is  incurred  when  the  server  switches  from  one  product  to 
another.  In  all  other  respects,  the  setup  cost  problem  and  the  setup  time  problem  are 
identical,  and  all  relevant  notation  from  §1  will  be  retained.  As  in  the  setup  cost  problem, 
we  shall  only  consider  dynamic  cyclic  policies.  Let  5  denote  the  mean  setup  time  per  cycle: 
Coffman,  Puhalskii  and  Reiman  (1995b)  show  that  the  heavy  traffic  performance  of  this 
system  depends  upon  the  setup  time  distributions  only  via  this  quantity,  and  other  key 
parameters  are  unaffected  by  a  permutation  in  the  cycle  order.  Thus  in  heavy  traffic 
the  desired  permutation  is  the  travelling  salesman  tour,  where  the  intercity  distances  are 
given  by  the  mean  setup  times.  Under  general  traffic  conditions,  Poisson  arrivals  and 
deterministic  service  times,  Fuhrmann  (1992)  and  Cooper,  Niu,  and  Srinivasan  (1992) 
show  that  the  mean  waiting  time  depends  on  setup  times  onl\'  through  the  total  setup 
time  over  the  cycle.  Also,  numerical  calculations  in  Federgruen  and  Katalan  (1993)  show- 
that  the  performance  of  a  cyclic  policy  is  quite  insensitive  to  the  chosen  permutation.  The 
server  has  three  scheduling  options  at  each  point  in  time:  Produce  a  unit  of  the  product 
that  is  currently  set  up,  initiate  a  setup  for  the  next  product  in  the  cycle  or  sit  idle.  The 
control  problem  is  to  find  a  nonanticipating  scheduling  policy  to  minimize 

\\msupl;E\rY.ih.!^{t)  +  b,i-{t))dt].  (.39) 

T_>co       i  '■Jo      ^   ^  ^         J 

Once  again,  we  avoid  unnecessary  notation  by  omitting  the  equations  that  express  the 
inventory  vector  /  in  terms  of  an  arbitrary  cyclic  policy. 


99 


2.2.  The  Diffusion  Control  Problem.  The  heavy  traffic  normalizations  are  the 
same  as  in  §2,  and  the  setup  times  are  not  rescaled  in  heavy  traffic;  that  is,  they  are  0(1). 
The  form  of  the  proposed  policy  is  also  the  same  as  before.  The  server  idles  if  W{t)  >  wq 
and  works  otherwise,  where  ioq  is  an  unknown  normalized  threshold  parameter.  When 
the  server  is  busy  and  the  total  workload  equals  w,  the  cycle  time  is  t{iu)  and  the  vector 
of  average  inventory  workloads  is  x'^(w). 

Since  the  setup  times  are  0(1),  they  occur  instantaneously  in  the  heavy  traffic  limit, 
and  Coffman,  Puhalskii  and  Reiman  (1995b)  show  that  the  time  scale  decomposition 
described  in  §1.3  also  carries  over  to  the  setup  time  problem.  More  specifically,  when  the 
total  normalized  workload  \'V{t)  equals  u',  the  ./V-dimensional  fluid  process  {Wi, . . . ,  Wj^j) 
is  identical  to  the  A'^-dimensional  fluid  process  in  the  setup  cost  problem;  hence,  the 
optimal  placement  of  cycle  center  x'^  is  given  by  equation  (19)  and  the  inventory  cost  rate 
c,(.r",r,  u')  is  defined  in  equations  (20)-(21). 

We  now  characterize  the  normalized  total  workload  process  W  for  the  setup  time 
problem  under  the  class  of  dynamic  cyclic  policies.  Our  characterization  of  the  total 
workload  process  follows  from  existing  heavy  traffic  limit  theorems  in  Coffman,  Puhalskii 
and  Reiman  (1995b)  after  making  the  innocuous  assumption  that  the  time  scale  decom- 
position holds  for  all  multi product  cyclic  policies,  not  just  for  two-product  cyclic  base 
stock  policies.  For  make-to-order  systems,  readers  are  referred  to  Coffman,  Puhalskii  and 
Reiman  ( 1995b)  for  a  derivation  of  this  characterization  in  the  two-product  uncontrollable 
case,  and  to  Reiman  and  Wein  for  a  discussion  of  the  controllable  case;  here  we  only  state 
the  approximation.  The  normalized  total  workload  process  W  is  assumed  to  be  a  diffusion 
process.  Its  variance  a^  is  the  same  as  in  the  setup  cost  problem,  and  is  defined  in  (2). 
The  drift  of  W  is  no  longer  v^(l  —  p).  but  x/'^(/(«')  —  p),  where  f{u')  is  the  fraction 
of  time  during  a  cycle  that  the  server  serves  customers,  as  opposed  to  incurring  setups, 
when  the  total  workload  equals  w.  When  the  normalized  workload  equals  lo,  this  fraction 
is  given  by 


'nri  w)  +  s 


where  t{w)  is  the  cycle  length,  which  is  both  deterministic  and  controllable.     Hence, 


\/n{f{iv)  —  p)  -^  c  —  s/t{w)  as  n  -^  oo,  and  we  assume  that  W  has  the  state-dependent 

drift 

;^(uO  =  c--i-.  (41) 

t(w) 

Notice  that  this  drift  approaches  the  drift  in  the  setup  cost  problem  as  the  setup  times 
vanish  or  the  cycle  length  becomes  arbitrarily  large. 

Define  the  cost  function  c(t,  w)  =  J2',=i  c,(r.  w),  where  the  individual  cost  components 
are  defined  in  (20)-(21).  The  approximating  diffusion  control  problem  is  to  choose  the 
state-dependent  cycle  length  t{w)  >  0  and  the  threshold  wq  to  minimize 
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up  ^E[j\{Ti\V(t)),Wit))dt],  (42) 


where  W  is  a  (i^i{w),a^)  reflected  diffusion  process  on  (  — ocwq]-  Hence,  the  controllable 
cycle  length  t(w)  affects  both  the  drift  i.i(w)  and  the  cost  c{t,  w)  in  a  nonlinear  fashion. 
2.3.  The  Optimality  Conditions.  Problem  (42)  involves  a  drift  control  t{w) 
and  a  singular  control  via  the  reflecting  barrier  wq.  Although  the  drift  is  unbounded 
as  t{w]  -^  0,  we  proceed  as  if  standard  arguments  apply  (Mandl  1968)  and  state  the 
Hamilton-Jacobi-Bellman  optimality  conditions: 


<r\.., 


min   {  c{t(w).  w)  -g+  [c-  -—     \"(w)  +  —V"(w)  }  =0     for  w  <  wo  (43) 

'■(u')>o  [  \        t{w)J  2  J 

and 

V'{w)  =  0     for  w  >  Wo,  (44) 

where  g  is  the  gain  and  V'(.r)  is  the  potential  (relative  value)  function.  If  a  solution 
{g.V{w),T{w).  Wq)  can  be  found  to  (43)-(44).  then  (r(«'),a'o)  is  the  optimal  solution 
to  (42),  g  is  the  optimal  long  run  average  cost  (independent  of  initial  state),  and  V{w) 
is  the  cost  incurred  under  the  optimal  policy  when  the  initial  state  is  w  minus  the  cost 
incurred  when  the  initial  state  is  some  reference  state,  which  we  take  to  be  Wq.  We  assume 
that  V  G  C^,  and  define  p{w)  =  V'{w);  this  assumption,  which  is  known  as  the  heuristic 
jjrinciple  of  smooth  fit  (Benes,  Shepp  and  Witsenhausen  1980),  is  often  imposed  when 
solving  diffusion  control  problems. 
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Using  the  workload  cutoff  levels  ivi  and  u'2  to  distinguish  among  regions  I,  II  and  III 
{wi  and  W2  are  unknown  at  this  point  and  are  not  given  by  (28)-(29)),  we  substitute  the 
three  forms  of  cost  function  c{t,iu)  into  (43)  to  obtain 


min   <  <fir(u')  +  h^w  -  g  +  ic ^--    p{iv)  +  -—p'{iu)  \  =  0    for  lui  <  w  <  wq,   (45) 
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(46) 


min   <  (3t{w)  -  bjwiv  -  g  +  (  c ^  )  p{iu)  +  -—p'(iv)  >  =  0    for  w  <  1V2,         (47 
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where  the  new  constant  ^5  is 
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Solving  the  first-order  optimality  conditions  for  t{w)  yields 
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for  10 1  <  w 


for  u;2  <  It'  <  ifi 


for  w  <  W2 


(49) 


Substituting  t'{iv)  into  (45)-(47),  we  obtain  the  nonlinear  ordinary  differential  equations 
(ODE's) 


\/~^isp(w)  +  hiMiv  —  g  +  cp{io)  +  —p'(io)  =  0         for  Wi  <  w  <  Wq,     (50) 


'2y/^2U^^  -  ^2Sp(w)  +  ^5W  +  Cp[w)  +  —p'{lo)  -  ^  =  0 

2 

-y-^sspiw)  -  bNW  -  g  +  cp{w)  +  —p'{iv)  =  0 


for  W2  <  w  <  Wi,     (51) 


for  iv  <  W2. 


(52 


2.4  Structural   Properties.      Before  pursuing  a  solution  to  the  diffusion  control 
problem,  we  derive  several  useful  structural  properties. 


Property  1.    If  102  —  —00  then 


p(w)  =  w  +  o(W)     as     w  — >■  —00;  (5o) 


if  u'2  /  —  oc  f/jen 

p(it')  = w  +  o(w)     as     IV  -^  — cxo.  (54) 

c 


The  derivation  of  this  property,  which  assumes  an  asymptotic  monotonicity  property, 
is  nearly  identical  to  that  in  the  Appendix  of  Reiman  and  Wein  (1994);  see  Markowitz 
(1995)  for  details. 

Property  2.     The  idhng  threshold  wq  is  greater  than  or  equal  to  it'i,  and  equality  holds 
if  and  only  if  h,  =  hj  for  all  i  and  j.  At  the  idling  threshold,  the  cycle  length  r"(u'o)  =  0 


if  wq  >  u'l  and  t'(wo)  =  y'6/G»'o  'Tft'o  =  "'i- 

Proof:   The  boundary  condition  p(wo)  —  0  and  equation  (49)  imply  that  t'(wo)  =  0  if 


Wq  >  it'i  and  r*(ft'o)  =  \/^4/^2Wo  if  u'o  <  u'l-  In  order  for  iuq  <  u'l,  r'(»'o)  must  satisfy 
the  cycle  placement  conditions  originally  specified  in  the  conditioning  of  equation  (19). 
We  can  rewrite  the  centering  condition  as 

\wo  -  l^T  (u'o)a,  •  -/I  +  tt'oQ,  •  72I  <  ^ •  (55) 

!  =  1 

Rearranging  terms,  this  inequality  is 

.    [ii,      ,  pn{1--pn)  i^f.. 

73<\/— (^4  + 7, )■  (5") 

V   V2  ^ 

Algebraic  manipulations  (see  Markowitz)  show  that  this  inecjuality  is  false  if  inventory 
holding  costs  are  not  identical  and  holds  at  equality  only  if  h,  —  hj  for  all  i  and  j.  In  the 
cost-symmetric  case,  the  placement  conditions  in  equation  (55)  are  satisfied  at  equality 
by  the  idling  threshold  u'o,  and  hence  wq  is  the  boundary  between  regions  (I)  and  (II); 
that  is,  »'o  =  U'l.  I 
Property  3.     u'2  =  — oc  if  and  only  if  b,  —  bj  for  all  i  and  j. 
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Derivation:  The  constant  ^3  equals  0  in  the  cost-symmetric  case.  This  fact  and  (49) 
imply  that  T'(tu)  is  unbounded  for  w  <  u'2.  which  would  violate  the  placement  conditions. 
Hence,  u'2  =  —00  in  the  cost-symmetric  case.  If  W2  =  — cxd  then  p(w)/w  -^  ^^. ** — - 
as  w  — >•  —00  by  Property  1,  and  t''{w)/w  ->  -J 7^  as  w  — >  -00  by  (49).   To  remain  in 
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region  (II),  the  placement  condition 


AT-l 


IV 


t'{w)Pn{1  -  pn 


-  V  T'{iu)a,  ■  7i  -I-  a»Q,  •  72I  <  r (57) 


1=1 


must  be  satisfied.   For  10  approaching  —00,  we  can  substitute  the  limit  of  t'[w)  and  by 
algebraic  manipulation  reduce  equation  (57)  to 

^    1^,Pn{1  -  Pn)  .  ,^^. 

73  <  \/^( T 74)-  (5^) 

V    V2  " 

As  in  Property  2,  this  inequality  is  false  if  backorder  costs  are  not  identical  and  holds  at 
equality  if  6,  =  bj  for  all  ;  and  j  (see  Markowitz  for  details).  | 

The  nonlinear  ODE's  in  (50)-(52)  do  not  appear  to  admit  a  closed  form  solution.  As 
in  Reiman  and  Wein,  we  pursue  both  an  approximate  analytical  solution  and  a  numerical 
solution.  The  approximate  analytical  solution  aims  to  compute  a  function  p[.r)  that  nearly 
satisfies  the  optimality  conditions  (44)  and  (50)-(52).  The  basic  idea  behind  our  method 
is  to  use  Taylor  series  expansions  to  linearize  the  square  root  terms  in  (50)-(52),  and  then 
to  use  Properties  1  through  3  to  paste  together  the  solutions  of  the  resulting  ODE's  so  as 
to  ensure  that  the  principle  of  smooth  fit  holds.  However,  the  scheduling  policy  arising 
from  this  analysis  did  not  perform  consistently  well  over  the  test  cases  considered  in  §3. 
Hence,  this  approximate  analysis  and  the  corresponding  numerical  results  are  not  given 
here,  and  the  interested  reader  is  referred  to  Markowitz  for  details. 

The  next  subsection  contains  insights  from  our  structural  results,  and  an  algorithmic 
procedure  for  solving  the  diffusion  control  problem  is  provided  in  §2.6. 

2.5.  Insights.  Because  a  closed  form  solution  is  not  obtained  for  the  setup  time 
problem,  it  is  more  difficult  to  develop  insights  into  the  behavior  of  the  optimal  solution. 
Nevertheless,  several  noteworthy  comparisons  can  be  made. 


Cost-Symmetry  vs.  Cost-Asymmetry.  The  solution  for  these  two  cases  are  surpris- 
ingly different;  see  the  graphs  entitled  "proposed"  in  Figures  5  and  6  in  §3.3.  In  the 
cost-symmetric  case  the  workload  stays  in  region  II,  and  the  cycle  length  at  the  idling 
threshold,  r(ii'o),  is  proportional  to  wq.  In  the  cost-asymmetric  case,  the  policy  is  char- 
acterized by  three  regions,  and  the  lot  size  approaches  zero  as  the  workload  approaches 
the  idling  threshold;  these  results  follow  from  Properties  2  and  3.  Evidently,  near  the  id- 
ling threshold,  small  lot  sizes  are  used  in  the  asymmetric  case  to  reduce  inventory  costs, 
whereas  this  option  is  not  available  in  the  cost-symmetric  case.  Finally,  by  Properties  1 
and  3  and  equation  (49),  as  the  current  total  workload  w  tends  to  —  oo,  the  cycle  length 
grows  with  \/^w  in  the  asymmetric  case  and  with  —w  in  the  symmetric  case.  Hence. 
when  backorders  are  large,  there  is  less  opportunity  to  reduce  inventory  costs  in  the  cost- 
symmetric  case,  and  larger  lot  sizes  prevail  in  order  to  reduce  the  amount  of  time  devoted 
to  setups. 

Setup  Costs  vs.  Setup  Times.  It  is  interesting  to  note  both  the  similarities  and 
differences  between  the  setup  cost  and  setup  time  problems:  The  behavior  of  the  proposed 
policies  for  both  problems  can  be  distinctly  broken  down  into  three  workload  regions 
(one  region,  respectively)  when  costs  are  asymmetric  (symmetric,  respectively).  For  both 
problems,  lot  sizes  are  state-dependent  and  inventory  is  focused  in  the  least  cost  products; 
moreover,  the  description  of  the  inventory  levels  at  the  switching  epochs  in  §1.8  carries  over 
to  the  setup  time  problem.  The  proposed  policies  for  the  two  problems  are  qualitatively 
similar  in  the  region  around  w  =  0  (i.e.,  region  II),  but  have  different  characteristics  in 
the  two  extreme  regions  (regions  I  and  III).  In  the  asymmetric  setup  cost  problem,  the 
cycle  time  r  remains  constant  throughout  these  two  regions.  In  the  asymmetric  setup 
time  problem,  the  cycle  time  contracts  to  zero  as  iv  approaches  the  idling  threshold  and 


grows  as  \/— ic  when  w  tends  to  —  oo. 

As  noted  in  Reiman  and  Wein,  the  two  problems  lead  to  qualitatively  different  solu- 
tions because  queueing  effects  cause  setup  times  to  consume  available  capacity  in  a  highly 
nonlinear  manner.  Therefore,  the  effective  cost  of  a  setup  time  is  woMoad-dependent  in 
the  setup  time  problem:  There  is  no  direct  penalty  for  a  set  up,  only  an  increased  probab- 
ility that  the  total  workload  will  fall.  As  the  total  workload  approaches  — oo,  many  items 
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are  backordered  and  the  effective  cost  of  a  setup  is  very  high;  thus,  the  scheduler  attempts 
to  use  the  capacity  efficiently  by  running  large  lot  sizes,  so  as  to  recover  from  the  low 
workload  level.  In  contrast,  as  the  total  workload  approaches  the  idling  threshold  ioq,  the 
effective  cost  of  a  setup  time  decreases,  and  the  scheduler  can  afford  to  employ  small  lot 
sizes  to  reduce  inventory  costs.  As  a  consequence  of  our  analysis,  it  is  clear  that  setup 
costs  should  not  be  used  as  a  surrogate  for  setup  times  in  the  SELSP;  unfortunately,  this 
practice  is  quite  common  in  the  deterministic  ELSP  literature. 

Setup  Times  vs.  No  Setup  Times.  Like  the  setup  cost  policy,  the  proposed  setup  time 
policy  is  a  generalization  of  Wein.  As  setup  times  vanish,  all  of  the  inventory  becomes 
stored  only  in  product  N.  Using  the  symmetric  cost  results,  the  optimal  idling  threshold 
Wo  goes  to  —  ln(  ^)o-^/('2c),  which  is  the  same  as  that  derived  by  Wein. 

2.6  An  Algorithmic  Solution.  Since  problem  (42)  cannot  be  solved  analytically, 
we  pursue  a  numerical  solution.  In  particular,  the  Markov  chain  approximation  technique 
developed  by  Kushner  (1977)  is  employed.  This  method  systematically  discretizes  both 
time  and  the  state  space,  and  approximates  a  diffusion  control  problem  by  a  control  prob- 
lem for  a  finite  state  Markov  chain.  Weak  convergence  methods  have  been  developed  by 
Kushner  and  his  colleagues  to  verify  that  the  controlled  Markov  chain  (and  its  corres- 
ponding optimal  cost)  approximates  arbitrarily  closely  the  controlled  diffusion  process 
(and  its  corresponding  optimal  cost);  we  refer  readers  to  Kushner  and  Dupuis  (1992)  for 
an  up-to-date  account  of  this  research  area,  and  retain  most  of  their  notation  for  ease  of 
reference.  It  is  worth  emphasizing  that  the  computational  complexity  of  the  algorithmic 
approacli  is  independent  of  the  number  of  products;  this  important  fact  is  due  to  the 
state  space  collapse  inherent  in  the  CPR  result,  which  leaves  us  with  a  one-dimensional 
diffusion  process,  and  our  optimization  of  the  cycle  center  in  (19). 

Before  describing  the  method,  we  introduce  a  slight  modification  to  the  heavy  traffic 
analysis  to  account  for  the  fact  that  setup  times  do  not  vanish  in  the  original  problem. 
The  cycle  length  t{w)  consists  of  the  time  devoted  to  processing  and  the  time  allocated 
to  setups.  In  the  fluid  scaling,  sj ^Jn  units  of  time  are  spent  setting  up  over  the  course  of 
a  cycle;  although  this  quantity  vanishes  in  the  limit,  we  include  it  in  our  analysis  as  an 
intended  refinement.  More  specifically,  we  replace  t[w)  by  t{w)  -\-  s/s/n. 


Let  h  denote  the  finite  difference  interval,  which  dictates  how  finely  both  the  state 
space  and  time  are  discretized.  One  can  consider  a  sequence  of  controlled  Markov  chains 
indexed  by  the  interval  h,  and  as  the  value  of  h  becomes  smaller  the  resulting  discrete 
time,  finite  state  Markov  chain  described  below  becomes  a  better  approximation  of  the 
controlled  diffusion  process. 

To  numerically  solve  (42),  we  need  to  confine  the  one-dimensional  diffusion  process 
W  to  a  bounded  region.  Since  W  resides  on  a  halfline,  the  state  space  of  the  controlled 
Markov  chain  will  be  {  — M,  —M  +  h, ...,  M  —  h,  M},  where  M  is  a  positive  integer  multiple 
of  h.  For  now  let  us  fix  the  idling  threshold  wq  (this  parameter  will  be  optimized  later  on) 
such  that  Wq  <  M  and  ioq  is  an  integer  multiple  of  h.  Hence,  the  Markov  chain  actually 
resides  in  {  —  A/,  —M  +  h,...,Wo  —  h,iUo}.  The  approximating  Markov  chain  has  nonzero 
transition  probabilities 

a'  +  2h(c-    ,   .f  ,  ^Y 


2(j2  +  2h 


c 


t(iu)  +  s/^ 


and 

P''iw.w-h] 


2a^  +  2h\c 


t{w)  +  s/^ 

on  the  interior  of  the  state  space,  and  the  time  intervals,  or  interpolation  intervals^  are  of 
length 


At 


h 


a'  +  h 


(61) 


T{w)  +  s/s/n 

Two  issues  need  to  be  addressed  to  obtain  our  approximating  controlled  Markov 
chain:  (i)  for  an  ergodic  cost  problem,  the  interpolation  interval  At''  must  be  independent 
of  the  state  w  and  control  t(w)  (see  Kushner  and  Dupuis,  page  209),  and  (ii)  the  behavior 
of  the  Markov  chain  at  the  boundary  states  w  =  —N  and  w  =  Wq.  To  deal  with  the  first 

issue,  we  define  Q^  =  a^  +  max^,  ^(^,1  h\c ,   ^  f  ;  /-    •   Since  the  smallest  nonzero  value 

of  t(io)  is  h,  we  let  Q''  =  a^  +  \ch  —  s\  ,  and  define  the  new  nonzero  interior  transition 
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probabilities 


(T^  +  2h(c-    ,     '  ,^)^ 

P\iv.w-h)     =     V        r(,.)^siv^)  ^g3^ 

2(5 


and 


[a^  +  h 
=  1  — 

and  the  new  interpolation  interval 


P'^iiiuw)  =  1  -  ^'     '    "     ;("-)+-/^  ^  ,  (64) 


A<'  =  ^  .  (65) 

Now  we  consider  the  boundary  states.  A  reflecting  boundary  is  employed  at  the  idling 
threshold.  However,  the  Markov  chain  appro.ximation  method  assumes  that  A^''  =  0 
for  a  reflecting  boundary  state.  Because  the  interpolation  interval  A/''  takes  on  a  value 
diff'erent  than  (65)  at  tt'o,  this  boundary  state  must  be  eliminated.  We  define  the  transition 
probability  (see  page  212  of  Kushner  and  Dupuis) 

P^iwo  -  h.  Wo  -  h)  =  1  -  P\wo  -  h,  wq  -  2h)  .  (66) 

We  also  impose  a  reflecting  boundary  at  state  —M,  and  define  the  transition  probability 

P^i-M  +  h.  -M  +  /O  =  1  -  P(-M  +  /?,,  -M  +  2h)  .  (67) 

Although  the  reflecting  barrier  at  —M  is  artificial  in  the  sense  that  P{  —  M,—M  +  h) 
would  be  positive  if  the  boundary  was  chosen  to  be  larger  than  M,  the  effect  of  this 
approximation  should  be  negligible  if  the  value  of  M  is  suflficiently  large,  and  consequently 
visited  sufficiently  infrequently.  In  our  implementation,  the  size  of  the  Markov  chain  is 
chosen  so  that  a  further  increase  in  M  does  not  change  the  optimal  solution  {t{w),wo).  In 
summary,  our  approximating  Markov  chain  has  state  space  {  — M  +  /?,  —  M  +  2/?, ...,  wq  — 
2/i,  u'o  —  /?},  interpolation  interval  defined  by  (65),  and  nonzero  transition  probabilities 


P''{w,y)  defined  by  (66)-(67)  and  P^(w,y)  =  P^{w,y)  otherwise,  where  P''{w,y)  are 
defined  in  equations  (6'2)-(64). 

The  dynamic  programming  optimality  equation  for  the  controlled  Markov  chain  is 
given  by  (see  equation  5.3  on  page  204  of  Kushner  and  Dupuis) 


Viw)  =  I 


Ey  PHw, y)V{y)  +  (6(r(u')  +  s/^)  +  h^io  -  g)Ai' 
Ey  P\io,  y)V(y)  +  iUriw)  +  s/,/1^.)  +  6u. 

Ey  PHw.y)V(y)  +  iUriw)  +  s/^)  -  bj^ic  -  g)At' 


for  u'l  <  lu  <  lUo 
for  i«2  ^  w  <  it'i 


for  —N<w<  ii'2 
(681 


and  the  Markov  chain  control  problem  can  be  solved  using  the  following  policy  improve- 
ment algorithm.  First,  we  choose  the  initial  policy:  T{iii)  =  ioq  —  w  for  w  <  wq  and 
arbitrary  Wq.  In  the  evaluation  step  of  the  algorithm,  a  generic  policy  {t{ui),  Wq)  is  evalu- 
ated (that  is,  V(w)  and  g  are  found)  recursively.  Since  the  Markov  chain  is  a  birth-death 
process,  the  stationary  probability  distribution  tt,^,  for  w  G  {  —  M  +  h M  —  h}  is 


7r„ 


0 


P''{k-h.,fc) 


_[_  Y^'('o~^  rTti'o  — /i    P    (k,k  —  h)  \ 


for  w  >  Wq  —  h 
for  w  <  Wo  —  h 

for  IV  =  Wq  —  h 


(69) 


and  the  gain  is 


iro  -  h 

g^       Y^      it„,c(t(w)  +  s/^/n,w). 

w=-\  +  h 


,70) 


We  set  V'(A/)   =  0,  so  that  V{w)   =  0  for  w  >  u'o,  by  equation  (44).     For  w   <   wq, 
equation  (68)  implies  that  V{w  —  h)  can  be  calculated  recursively  by 


V(w-h: 


g-c{T(w),w)  +  {l  -  P''(w,iv)]V(w]  -  P''{w,w  +  h)V{iu  +  h] 

PHw.w-  h) 


(7i; 


In  the  policy  improvement  step  we  first  solve  for  the  cycle  length  t{w)  and  then  for  the 
idling  threshold  wq.  The  cycle  length  is  determined  by  minimizing  the  right  side  of  (68) 
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w 


ith  respect  to  t{iv)  >  0.  If  the  drift  c  —  s/{t'(id)  +  s/y/ri.)  is  positive  then 


t'{w)  = 


slV(w+h)-V{w)] 


il,n2  _   s\V{w+h)-V(u,)] 


slV{w+h)-V(w)] 


for  Wi  <  w  <  u'o 


for  IV2  <  u'  <  u'l       , 


for  —  A'^  <  iv  <  102 


(72) 


and  if  the  drift  is  negative  then 


r'iw) 


s\V(w)-V(w-h)] 


U      2         s[V{w)-V{w-h)] 


s[V{w)-V{w-h)] 


for  U'l  <  w  <  Wq 
for  W2  <  it-1  <  lUi 
iov  —N  <  u'  <  W2 


(73) 


Notice  that  (72)-(73)  converges  to  (49)  as  /?  — )•  0,  as  expected.  Keeping  t{w)  constant, 
we  determine  an  improved  idHng  threshold  wq  by  evaluating  (via  equation  (70))  the  gain 
g  for  all  values  of  wq,  and  searching  for  the  gain-minimizing  value. 

The  evaluation  and  improvement  steps  of  the  policy  improvement  algorithm  are  re- 
peated until  the  new  and  old  gains  are  sufficiently  close  in  value  and  the  idling  threshold 
remains  constant.  In  the  implementation  of  this  algorithm  in  §3,  we  chose  M  =  1000 
and  terminated  the  algorithm  when  the  difference  between  the  new  and  old  gains  was 
less  than  0.05  and  the  idling  thresholds  changed  less  than  0.1;  our  choice  for  the  finite 
difference  interval  h  is  discussed  in  the  next  paragraph.  The  output  of  our  algorithmic 
procedure  includes  the  workload  cutoffs  wq,  u'l  and  102,  as  well  as  t(w)  and  V(w)  for 
w  =  {  —  M  +  h, . . . ,  Wq  —  /?}.  In  the  next  subsection,  we  show  how  to  translate  this  output 
into  a  proposed  solution. 

2.7.  The  Proposed  Policy.  The  mapping  from  a  diffusion  control  solution  to  a 
proposed  policy  is  less  straightforward  when  a  numerical  solution  is  obtained  than  when 
an  analytical  solution  is  derived.  More  specifically,  with  a  numerical  solution  there  is  no 
way  to  develop  a  proposed  scheduling  policy  that  is  independent  of  the  heavy  traffic  scaling 
parameter  7j:  The  drift  of  the  underlying  diffusion  process  is  \/n.{l  —  p)  —  s/{T{w)  +  s/\/n), 


and  a  value  of  n  must  be  chosen  in  order  to  compute  a  numerical  solution  to  the  Markov 
chain  control  problem.  This  quandary  is  dealt  with  in  the  most  natural  way:  We  set  c 
ecjual  to  one  and  let  n  ~  {1  —  p)~^.  Moreover,  we  set  the  finite  difference  interval  h  equal 
to  l/\/n,  so  that  the  discretization  in  the  Markov  chain  approximation  procedure  corres- 
ponds to  individual  units  of  inventory  in  the  original  problem.  Exploratory  computations 
revealed  that  the  performance  of  the  proposed  policy  was  very  insensitive  to  our  choice 

of  77. 

Recall  that  vve  replaced  the  cycle  lengih  t{w)  by  t{io)  -\-sI y/n  in  the  Markov  chain  al- 
gorithm. Therefore,  in  creating  our  proposed  policy,  we  employ  the  cycle  center  Xc[t{w)-\- 
s/y/n^iu).  The  proposed  policy  for  the  setup  cost  case  produced  product  7  until  its  in- 
ventory reached  x1{t{iv),w)  -\-  f(iy)/9,(l  —  p,)/'2.  In  order  for  the  expected  total  busy 
time  in  a  cycle  to  be  equal  to  r{ih),  the  s/\/n.  term  is  not  added  in  the  second  of 
the  two  terms  in  this  expression,  and  we  produce  product  i  until  its  inventory  reaches 
.rUr(w)  +  s(l-  />),  Cu)  +  f(w)pAi  -  p,)/2. 

The  proposed  policy  is  constructed  just  as  in  the  setup  cost  problem.  Define  the 
unnormalized  workload  regions  I,  II  and  III  according  to  whether  the  quantity  (1  — 
p)J2'i=iP7^f>{0  is  greater  than  u'l,  in  the  interval  [77'2,7i'i]  or  less  than  tt'2,  respectively. 
Then  our  proposed  policy  is:  If  (I  —  p)  X]i=i  p7^  MO  >  0  f^^"  ^^^  ^  refer  to  the  product 
with  the  smallest  holding  cost  index  h,:  otherwise,  let  N  denote  the  product  with  the 
smallest  value  of  b,.  The  server  should  idle  if{\  —  p)  Yl,^=\  P7^  U[^)  >  Wq;  otherwise,  if  set 
up  for  product  i  <  N  then  produce  this  product  as  long  as 
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Once  jj.^  '/,(0  reaches  or  exceeds  this  level,  switch  to  the  next  product.    If  set  up  for 
product  i'V,  then  produce  this  product  while 
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and  then  switch  to  the  next  product  when  i^ijj^ I[^{t)  reaches  or  exceeds  this  level. 

The  solution  proposed  above  is  specified  in  terms  of  the  original  problem  parameters, 
and  the  constants  u'o,  u'l,  u'2  and  the  function  t(w)  generated  by  the  algorithmic  proced- 
ure. Since  t(w)  is  only  defined  on  a  discrete  state  space,  the  argument  of  this  function 
is  rounded  to  the  closest  discrete  value  in  the  algorithmic  discretization;  that  is,  t{w)  is 
approximated  by  r(y*),  where  y' satisfies  |y*  —  u;|  <  |y  — lol  for  all  y  G  [—N  +  h, . . .  ,Wo  —  h]. 


3.    COMPUTATIONAL  STUDY 

In  this  section  we  evaluate  the  effectiveness  of  our  proposed  policies  by  conducting 
a  series  of  two-product  and  five-product  experiments  for  both  the  setup  cost  and  setup 
time  problems.  For  the  two-product  cases,  we  compare  the  performance  of  our  proposed 
policy  and  two  "straw'  policies  against  a  numerically  derived  optimal  policy.  A  dynamic 
programming  value  iteration  algorithm  is  used  to  find  the  optimal  policy  and  evaluate  the 
performance  of  all  four  policies.  ^From  this  data  we  compute  the  suboptimality  for  the 
proposed  and  two  straw  policies  by 

.  policy's  cost  —  optimal  cost 

policy  s  suboptmiaiitv  =  : ; x  lOO/o  . 

optimal  cost 

In  implementing  the  value  iteration  algorithm  (readers  are  referred  to  Markowitz  for 
a  detailed  specification  of  the  algorithm),  the  inventory  state  space  was  truncated  to 
[-150.150]  by  [-150,150]  for  the  setup  cost  cases  and  [-250,250]  by  [-250,250]  for  the 
setup  time  cases.  To  achieve  three-digit  accuracy  of  the  suboptimalities,  7,000  iterations  of 
the  algorithm  were  required  for  the  setup  cost  problem  and  14.000  for  the  setup  time  prob- 
lem. Due  to  the  large  number  of  inventory  states,  a  dynamic  programming  algorithm  is 
not  feasible  for  the  five-product  cases,  and  thus  no  optimal  policy  is  derived.  Instead,  dis- 
crete event  singulation  is  used  to  evaluate  the  proposed  policy  and  the  two  straw  policies. 
To  evaluate  a  policy  for  a  particular  scenario,  we  perform  5  independent  runs  of  6,000,000 
time  units  for  the  setup  cost  problems  and  10  independent  runs  of  6,000,000  time  units 
for  the  setup  time  problems;  each  run  starts  with  an  empty  system  and  statistics  from  the 
first  10,000  time  units  are  discarded.  For  all  scenarios  in  this  section,  we  assume  that  the 
demand  interarrival  times,  service  times  and  setup  times  are  exponentially  distributed: 
service  is  preemptive  in  the  two-product  cases  and  non-preemptive  in  the  five-product 
cases. 

For  systems  with  two  products,  we  consider  20  setup  cost  cases  and  14  setup  time 
cases;  all  but  two  cases  for  each  type  of  problem  assume  that  the  products  have  identical 
parameters,  .'\lthough  nearly  all  of  our  cases  are  symmetric,  the  numerical  results  in 
Reiman  and  Wein  suggest  that  the  heavy  traffic  analysis  is  ecjually  accurate  for  symmetric 
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and  asymmetric  problems.  For  systems  with  five  products,  we  consider  six  setup  cost  cases 
and  four  setup  time  cases.  We  focus  on  the  two-product  setting  for  several  reasons.  The 
optimal  solution  can  be  numerically  computed  in  this  setting,  which  allows  us  to  assess 
the  suboptimality  of  our  proposed  policies;  since  the  optimal  policy  is  a  dynamic  cyclic 
policy  in  the  two-product  case  (i.e.,  the  optimal  policy  chooses  one  of  the  three  scheduling 
options  that  we  allow  at  each  point  in  time),  we  conjecture  that  our  proposed  policies  are 
optimal  in  the  heavy  traffic  limit.  Also,  the  graphical  depictions  of  the  various  policies 
in  two  dimensions  (see  Figures  2  through  6)  help  us  to  understand  the  subtleties  of  the 
behavior  of  this  system.  The  two  straw  policies  are  described  in  §3.1,  and  the  numerical 
results  for  the  setup  cost  and  setup  time  problems  are  given  in  §.3.2  and  §3.3,  respectively. 
Our  key  observations  are  summarized  in  §3.4. 

3.1  Straw  Policies.  To  help  assess  the  effectiveness  of  the  proposed  policy,  we 
consider  two  simpler  classes  of  cyclic  policies,  and  use  heavy  traffic  analysis  to  optimize 
within  these  classes.  One  is  a  generalized  base  stock  policy  and  the  other  is  a  fi.xed  size 
corridor  policy  similar  to  one  considered  by  Sharifnia,  Caramanis  and  Gershwin.  Neither 
straw  policy  employs  the  s/^/ri  refinement  that  was  introduced  in  §2.7;  we  discuss  this 
issue  in  §3.4. 

Generalized  Base  Stock  Policy.  The  generalized  base  stock  policy  can  be  stated 
as  follows.  If  the  server  is  set  up  for  product  i,  then  serve  this  product  if  W,(t)  <  v,.  If 
^Vi{t)  >  f'i,  then  idle  if  product  j,  the  next  product  to  be  produced  in  the  cycle,  has  a 
workload  level  Wj(t)  >  Vj  —  yj\  otherwise,  switch  to  product  j  at  this  point.  Hofri  and 
Ross  (1987)  prove  that  the  make-to-order  version  of  this  policy  is  optimal  in  a  two-product 
symmetric  polling  system.  The  generalized  base  stock  policy  can  be  thought  of  as  a  refined 
version  of  the  cyclic  base  stock  policy  considered  by  Federgruen  and  Katalan  ( 1993),  in  the 
sense  that  their  policy  can  only  insert  idleness  in  a  state-independent  manner.  Although 
the  generalized  base  stock  policy  contains  2A'^  parameters,  the  heavy  traffic  behavior  of 
this  policy  (see  Reiman  and  Wein  for  details)  depends  on  the  yj's  only  via  maxi<;<;v  yn 
let  us  denote  this  quantity  by  y.  Hence,  we  set  each  y,  equal  to  y,  and  optimize  over 
the  iV  -|-  1  normalized  parameters  (ci, . . . ,  vj^.y),  where  y  =  y/\/n  and  v,  =  vj\/n.  In 
heavy  traffic,  this  policy  is  equivalent  to  one  that  completes  production  of  product  /'  when 


its  inventory  level  reaches  Vi,  and  employs  the  workload  idling  threshold  X],=i  *•'!  ~  y\  see 
Figure  2. 

To  calculate  the  cost  associated  with  this  policy,  we  move  from  these  natural  para- 
meters to  those  used  in  §2  and  §3.  I'nder  the  CPR  results,  for  a  given  total  workload  w 
a  generalized  base  stock  policy  has  cycle  center 

j,(u')  =  V,  -  — ^ -(2^  Vj  -  w)  (/6) 

E;=i^;(l  -  Pi)  j=i 


=1  -'t       ^' 


and  cycle  length 

t{iu)  =  2    ^'='" ^.  (77) 

Product  Vs  average  inventory  cost  is  obtained  by  substituting  these  parameters  into  equa- 

tion  (4);  I.e.,  c,(2    V"' v, ^n        :'     ,      ,  u'  ■ 

Using  the  CPR  results,  we  can  derive  the  total  average  cost  for  the  generalized  base 
stock  policy  for  both  the  setup  cost  and  time  problems.  In  the  setup  cost  case,  total  aver- 
age cost  is  calculated  by  integrating  the  average  inventory  costs  plus  average  setup  costs 
over  the  stationary  distribution  of  the  normalized  total  workload.  Since  the  normalized 
total  workload  W  is  approximated  by  a  RBM,  the  total  average  cost  is 


/£- "-'  e;I,  c,(2^^.;;- ■ .,  -  ^■^^-:;!^^Uy\ ,,) 

where  a  =  2>/n(l  —  p)/cr^.  For  the  setup  time  problem,  the  average  inventory  cost  is 
similar,  although  the  stationary  distribution  of  W  is  no  longer  exponential,  but  gamma 
(see  Coffman,  Puhalskii  and  Reiman  1995b).  The  total  average  cost  is 

/S"'  '■-"  (el  .(2*^^.  =v  -  -'^"^p-r'. ...)) 

V  E,  =  ,''-<1-''')  E;=,^;<>-Pj)  J  (79) 

"(°(E.l,".-))%,-a(E;:.-'.-'^-)^^, 

where  q  is  as  above  and  f3  =  sJ2^=\Pi{^  —  Pi)/^^-  ^Ve  set  n  equal  to  (1  —  p)~^  and 
use  a  steepest  descent  algorithm  to  find  the  parameters  (I'l, .  .  .  ,  I'/v,  y)  that  minimize  (78) 
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Generalized  Base  Stock  Policy 


Figure  2:  The  two  straw  policies. 


and  (79).     For  both  the  setup  cost  and  setup  time  cases,  we  reverse  the  heavy  traffic 
scaling  to  obtain  the  proposed  parameter  values  Vt  =  vj(l  —  p)  and  y  =  i//(l  —  p). 

The  Corridor  Policy.  This  policy  can  be  stated  in  terms  of  switching  hyperplanes 
in  the  product  workload  space.  The  hyperplanes  are  created  to  form  a  fixed  width  corridor 
with  its  long  axis  orthogonal  to  the  constant  workload  plane  (see  Figure  2).  The  policy 
represents  a  natural  embodiment  of  the  "constant  lot  size"  philosophy  within  a  dynamic 
stochastic  framework,  and  is  defined  by  ^  +  2  parameters:  The  cycle  length  r  (or  corridor 
width),  the  idling  threshold  Wq  and  the  parameters  (yi, . .  .  ,yAf),  which  determine  the 
intercept  of  the  corridor's  axis.  We  can  use  these  variables  and  the  notation  of  the 
previous  two  sections  to  formulate  the  average  inventory  cost  of  the  policy.  For  a  given 
workload  i«,  the  cycle  center  .r^  is  equal  to  wj N  +  y,  and  the  cycle  length  is  r.  Product 
Ts  average  inventory  cost  for  workload  w  is  then  c,(t,w/N  +  yi,w). 

Because  the  cycle  length  is  independent  of  workload,  in  both  the  setup  cost  and  setup 
time  cases  the  diffusion  process  W  is  approximated  by  a  RBM  on  (  — cxd,u'o],  which  has 
an  exponential  steady  state  distribution.  The  drifts  of  the  RBM  are  c  and  c  —  sjr  for  the 


setup  cost  and  time  problems,  respectively.  Thus,  the  total  average  cost  is 


/ 


Ci(^.  "IT +  yn"')  +  -     -. e        -=  'dw  (bO) 


oo 


for  the  setup  cost  problem,  and 


/ 


'"'     X-     I      «'  ,^  2(v/^(l  -p)-  s/t)   _AV^[^-P)-'lr)^^,^_^,^ 


w=i 


for  the  setup  time  problem.  The  cost-minimizing  parameters  for  (80)  and  (81)  are  de- 
termined by  a  steepest  descent  algorithm.  .Although  we  were  able  to  use  77  =  (1  —  p)~-^ 
in  our  computations,  in  the  setup  time  problem  one  must  be  careful  to  choose  the  heavy 
traffic  scaling  factor  77  so  that  \/r?(l  —  p)  is  greater  than  sjr,  thereby  guaranteeing  a  well 
defined  integral;  this  inequality  is  simply  the  stability  condition  that  the  fraction  of  time 
the  server  spends  processing  units  and  setting  up  must  be  less  than  one.  Finally,  the 
proposed  parameter  values  are  given  by  y,  =  y,/(l  —  />)  and  f  =  r/(l  —  p). 

3.2.  The  Setup  Cost  Problem. 

Two-Product  Cases.  To  standardize  the  two-product  scenarios,  we  set  the  service 
rates  //i  =  ^2  =  1  and  control  the  utilization  rates  p,  by  varying  the  demand  rates  A,.  We 
also  set  /?2  =  1  and,  by  modifying  /i),  b\  and  62,  select  product  2  as  the  least  cost  product. 
Inventory  costs  and  arrival  rates  are  identical  across  products  in  the  IS  synj/iietric  cases, 
and  each  case  is  characterized  by  three  parameters:  Backorder  cost,  traffic  intensity  and 
setup  cost  per  cycle.  We  examine  all  permutations  of  values  shown  in  Table  I;  notice  that 
some  of  these  scenarios  grossly  violate  the  heavy  traffic  conditions.  The  parameters  for 
the  first  asymmetric  case  are  Ai  =  0.6,  A2  =  0.3,  /7i  =  2,  61  =  10,  62  =  5  and  A'  =  200. 
The  second  asymmetric  case  is  the  same  as  the  first,  except  that  the  backorder  cost  is 
doubled  to  6,  =  20  and  62  =  10. 

Table  II  displays  the  results  for  the  20  two-product  cases  and  Tables  III  to  V  show 
the  averages  (for  the  18  symmetric  cases)  over  individual  parameters  for  each  policy.  The 
switching  curves  for  the  optimal  and  proposed  policies  for  the  (6  =  5,  A'  =  200,  p  =  0.9) 
two-product  symmetric  case  is  depicted  in  Figure  .3,  and  corresponding  curves  for  the 
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Backorder  Cost     Setup  Cost     Traffic  Intensity 
b A^ p 


10 


20 

0.5 

100 

0.7 

200 

0.9 

Table  I:  The  18  test  cases  for  the  symmetric  two-product  setup  cost  problem. 
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Figure  3:  Switching  curves  for  a  symmetric  setup  cost  case. 


61  =  10  asymmetric  case  are  displayed  in  Figure  4. 

Five-Product  Cases.    We  set  A,  =  0.18  and  /^i,  =  1  for  ;  =  1 5  for  each  of  the 

six  cases,  resulting  in  a  traffic  intensity  of  0.9.   We  also  set  6,  =  ^h,  for  /  =  1 5  for 

half  the  cases  and  6,  =  lO/i,  for  the  other  half.  Each  case  is  characterized  by  /?,,  6,  and 
the  setup  cost.  Four  of  the  six  cases  are  symmetric  (/z,  =  1  for  /  =  1, .  .  .  ,-5)  and  two  of 
the  six  cases  are  asymmetric  (/?,  =  ;  for  /  =  1, . . .  ,5).  The  average  cost  for  each  policy 
(along  with  9.5%  confidence  intervals)  is  displayed  in  the  first  six  rows  of  Table  VI. 

3.3  The  Setup  Time  Problem.  .\s  in  the  two-product  setup  cost  test  cases,  we 
assume  that  //i   —  /l/2  =   1  and  /?2  =  1.    In  the  12  symmetric  scenarios,  each  product's 
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20 

0.5 

4.30 

6.0% 

6.5% 

9.6% 
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20 

0.7 

6.73 

8.3% 

1.5% 

20.1% 

5 

20 

0.9 

17.99 

3.8% 

7.5% 

26.8% 

5 

100 

0.5 

7.00 

6.2% 

6.4% 

18.5% 

5 

100 

0.7 

9.72 

2.1% 

2.6% 

18.5% 

5 

100 

0.9 

20.14 

1.5% 

8.1% 

27.4% 

5 

200 

0.5 

9.14 

15.0% 

7.2% 

25.3% 

5 

200 

0.7 

12.24 

2.6% 

2.9% 

21.3% 

5 

200 

0.9 

22.22 

0.7% 

7.5% 

26.1% 

10 

20 

0.5 

5.30 

14.1% 

2.2% 

20.0% 

10 

20 

0.7 

8.41 

13.9% 

4.2% 

24.6% 

10 

20 

0.9 

23.58 

6.0% 

11.7% 

33.0% 

10 

100 

0.5 

7.98 

8.0% 

6.1% 

15.5% 

10 

100 

0.7 

11.31 

4.5% 

3.7% 

25.5% 

10 

100 

0.9 

25.44 

3.3% 

9.6% 

34.9% 

10 

200 

0.5 

10.21 

6.4% 

7.2% 

18.6% 

10 

200 

0.7 

13.79 

3..5% 

4.7% 

24.3% 

10 

200 

0.9 

27.26 

2.1% 

10.0% 

35.0% 

Asym. 

6i  =  10 

28.71 

2.6% 

13.1% 

35.8% 

Asym. 

6,  =20 

35.76 

3.4% 

14.9% 

45.8% 

Table  II:  Results  for  the  two-product  setup  cost  cases. 

inventory  costs  and  service  utilizations  are  identical  and  we  vary  only  the  backorder  cost, 
the  traffic  intensity  and  the  ax'erage  setup  time  per  cycle.  Table  VII  reports  all  of  the 
permutations  of  values  analyzed.  The  first  asymmetric  scenario  is  defined  by  \\  —  0.6, 
A2  =  0.3,  //I  =  ^-2  =  I.  hx  =  2,  /?.2  =  1.  61  =  10,  62  =  5  and  5  =  20.  The  second 
asymmetric  scenario  is  identical  except  that  the  backorder  costs  are  61  =  20  and  62  =  10. 
The  individual  results  for  the  14  runs  are  displayed  in  Table  VIII  and  policy  summaries 


Backorder  Cost     Setup  Cost     Traffic  Intensity 
b K p 


Low 

Medium 
High 


5.1% 
6.9% 


8.7% 
4.3% 
5.0% 


9.3% 
5.8% 
2.9% 


Overall  Average  Suboptimality  =  6.0% 


Table  III:  .Average  suboptimality  of  the  proposed  policy:  Setup  cost  problem. 
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5.6% 

5.9% 

6.1% 

3.3% 

6.6% 

9.1% 

Backorder  Cost     Setup  Cost     Traffic  Intensity 
b A^ I) 

Low  5.6% 

Medium 

High  6.6% 

Overall  Average  Suboptimality  =  6.1% 
Table  IV:  Average  suboptimality  of  the  corridor  policy:  Setup  cost  problem. 


Backorder  Cost     Setup  Cost     Traffic  Intensity 
b A^ p 


Low  21.5%  22.4%  17.9% 

Medium  23.4%  22.4% 

High  25.7%  25.1%  30.5% 

Overall  Average  Suboptimality  =  23.6% 
Table  V:  Average  suboptimality  of  the  generalized  base  stock  policy:  Setup  cost  problem. 

for  the  12  symmetric  runs  are  given  in  Tables  IX  to  XI.  In  addition,  Figures  5  and  6  provide 
a  graphical  depiction  of  the  proposed  and  optimal  policies  for  a  symmetric  case  (6  =  5, 
s  =  2,  /9  =  0.9)  and  the  6i  =  10  asymmetric  case,  respectively. 

Results  for  two  five-product  scenarios  can  be  found  in  Table  VI;  they  are  identical 
to  the  setup  cost  scenarios  described  in  §3.2,  except  that  setup  times  (with  .s  =  50)  are 
incurred  rather  than  setup  costs. 

3.4.  Observations.  Our  observations  from  the  numerical  results  are  summarized 
in  this  subsection.  The  five-product  cases  are  discussed  after  the  two-product  cases. 

Performance  of  the  proposed  policy:  In  the  setup  cost  cases,  the  proposed  policy's 
average  suboptimality  is  6.0%  over  the  18  symmetric  scenarios.  The  policy  performs  very 
well  when  the  heavy  traffic  conditions  are  satisfied;  for  example,  the  suboptimality  is 
0.7%i  when  bi  =  5,  A'  =  200  and  p  =  0.9.  Considering  that  the  proposed  policy  was 
constructed  via  a  heavy  traffic  approximation,  it  operates  reasonably  well  over  a  wide 
range  of  system  parameters,  including  a  low  utilization  rate  of  0.5.  Not  surprisingly,  the 
policy  performs  worst  when  the  traffic  intensity  is  low,  the  setup  costs  are  small,  and  the 
backorder  costs  are  high.  The  policy  also  performs  well  (2.6%  and  3.4%  suboptimalities) 
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Figure  4:  Switching  curves  for  the  61  =  10  asymmetric  setup  cost  case. 

in  the  asymmetric  cases. 

In  the  setup  time  cases,  the  average  suboptimality  over  the  12  symmetric  cases  is  7.2%. 
The  policy  performs  very  well  (1.8%  average  suboptimality)  when  the  traffic  intensity  is 
high,  but  degrades  somewhat  in  the  lighter  traffic  cases.  It  also  performs  well  in  the 
asymmetric  cases  (1.5%)  and  3. 3%-  suboptimalities). 

Switching  curves:  The  switching  curves  of  the  proposed  and  optimal  policies  are 
remarkably  similar  in  Figures  3  to  6  and  are  unlike  either  the  corridor  or  generalized  base 
stock  policies.  In  the  two  symmetric  problems  (Figures  3  and  5),  these  curves  have  the 
same  general  shape  as  predicted  by  our  heavy  traffic  analysis:  A  distinctive  constant- 
workload  idling  threshold,  a  wide  cycle  time  for  large  positive  and  negative  inventories 
and  a  small  cycle  time  about  the  zero  total  workload  level.  In  the  asymmetric  setup  cost 
problem  in  Figure  4,  the  three- region  categorization  predicted  by  the  heavy  traffic  theory 
is  easily  recognizable  in  the  optimal  policy.  Figure  6  confirms  that  lot  sizes  shrink  as  the 
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Setup 

Cost  of 

Cost  of 

Cost  of 

order 

Cost  or 

Cost 

Proposed 

Corridor 

Gen.  Base  Stock 

Cost 

Time 

Structure 

Policy 

Policy 

Policy 

6  =  5 

A'  =  50 

Symmetric 

25.32(±0.46) 

2S.78(±0.63) 

33.23(±0.45) 

6  =  5 

A'  =  500 

Symmetric 

37.23(±0.10) 

37.25(±0.32) 

44.02(±0.20) 

6=  10 

A'  =  50 

Symmetric 

36.79(±0.73) 

39.39  ±0.91) 

40.54(±0.68 

6=  10 

A'  =  500 

Symmetric 

47.08(±0.38) 

46.02(±0.27) 

54.29(±0.52) 

6  =  5 

A'  =  500 

Asymmetric 

79.91(±0.41) 

86.65(±0.67) 

121.11(±1.57) 

6=  10 

A'  =  500 

Asymmetric 

98.46(±0.77) 

105.98(±1.49) 

138.88(±1.14) 

6  =  5 

.s  =  50 

Symmetric 

215.4(±4.9) 

228.0(±16.1) 

214.1(±2.6) 

6=  10 

5  =  50 

Symmetric 

264.7(±10.4) 

532.5(±  136.8) 

260.2(±4.7) 

6  =  5 

s  =  50 

Asymmetric 

610.8(±8.9) 

683. 9(  ±35.2) 

661.0(±9.1) 

6=  10 

.s  =  50 

Asymmetric 

737.4(±18.7) 

827.9(±66.5) 

791.7(±16.1) 

Table  VI:  Results  for  the  five-product  cases. 


Backorder  Cost     Setup  Time     Traffic  Intensity 
b  s  p 


Low 

5 

2 

0.5 

Medium 

0.7 

High 

10 

20 

0.9 

Table  VII:  The  12  test  cases  for  the  symmetric  setup  time  problem. 


idling  threshold  is  approached.  Pinally,  as  the  total  workload  ib  tends  to  minus  infinity, 
lot  sizes  appear  to  be  growing  roughly  with  —w  in  Figure  5  and  with  \/—w  in  Figure  6. 
Two  key  differences  between  the  proposed  and  optimal  policies  emerge  from  study- 
ing Figures  3  to  6;  numerical  results  (not  reported  here)  verify  that  both  discrepancies 
dissipate  as  the  traffic  intensity  approaches  unity,  and  get  more  severe  in  the  lower  utiliz- 
ation cases.  First,  in  all  four  figures,  the  proposed  heavy  traffic  policies  have  a  tendency 
to  backorder  more  than  the  optimal  policy;  this  observation  is  most  obvious  in  the  up- 
per right  portion  of  Figure  5.  Because  CPR's  time  scale  decomposition  does  not  hold 
precisely  for  the  original  stochastic  system,  the  optimal  policy  hedges  against  backorders 
slightly  more  than  the  proposed  heavy  traffic  policy,  which  assumes  that  the  inventory 
levels  respond  in  a  deterministic  fashion  in  the  fluid  limit.  In  terms  of  these  figures,  the 
cycle  lengths  (i.e.,  the  distance  along  the  total  workload  line  between  the  solid  and  dashed 
curves)  tend  to  be  slightly  smaller  in  the  optimal  policy;  consequently,  the  workload  pro- 


Backorder      Setup        Traffic       Optimal  Suboptimality  of 

Cost           Time      Intensity       Policy       Proposed            Corridor            Gen.  Base  Stock 
_b ^ () Gain Policy Policy Policy 


5 

2 

0.5 

3.84 

9.0% 

12.8% 

6.6% 

5 

2 

0.7 

7.45 

7.4% 

8.3% 

7.8% 

5 

2 

0.9 

25.31 

1.4% 

6.7% 

9.5% 

5 

20 

0.5 

13.58 

10.1% 

24.3% 

19.9% 

5 

20 

0.7 

26.39 

5.9% 

16.0% 

9.6% 

5 

20 

0.9 

79.40 

1.4% 

8.7% 

3.5% 

10 

2 

0.5 

5.15 

9.3% 

11.0% 

11.8% 

10 

2 

0.7 

9.85 

8.2% 

7.1% 

9.0% 

10 

2 

0.9 

33.24 

1.8% 

4..5% 

13.7% 

10 

20 

0.5 

17.75 

18.7% 

26.5% 

17.9% 

10 

20 

0.7 

33.67 

10.7% 

16.8% 

12.8% 

10 

20 

0.9 

98.93 

2.6% 

7.0% 

5.9% 

Asym. 

6i  =  10 

104.91 

1.5% 

30.6% 

11.5% 

Asym. 

6i=20 

129.42 

3.3% 

60.9% 

15.8% 

Table  VIII:  Results  for  the  two-product  setup  time  cases. 

Backorder  Cost     Setup  Time     Traffic  Intensity 
6 s 2 

Low  5.9%  6.2%  11.8% 

Medium  8.1% 

High  8.5%  8.2%  1.8% 

Overall  .Average  Suboptimality  =  7.2%i 
Table  LX:  Average  suboptimality  of  the  proposed  policy:  Setup  time  problem. 

cess  spends  less  time  in  the  backorder  region  and  some  of  our  remarks  in  §2.5  regarding 
the  inventory  levels  at  switching  epochs  only  hold  in  very  heavy  traffic.  This  limitation 
of  the  heavy  traffic  theory  was  also  noted  in  Wein. 

The  other  main  discrepancy  occurs  near  the  idling  threshold  in  the  asymmetric  cases: 
In  Figure  4,  the  optimal  lot  sizes  for  product  1  decrease,  rather  than  stay  constant,  as  the 
workload  idling  threshold  is  approached,  and  in  Figures  4  and  6  there  is  a  different  idling 
threshold  for  each  product.  This  discrepancy  can  be  explained  as  follows:  When  the  total 
workload  is  positive,  the  switchover  cost  (in  Figure  4)  or  time  (in  Figure  6)  makes  it 
beneficial  to  be  set  up  for  product  1,  so  as  to  efficiently  protect  against  costly  product 
1  backorders.   If  p  is  not  close  to  one,  then  it  is  likely  that  the  total  inventory  workload 
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Backorder  Cost     Setup  Time     Traffic  Intensity 
b ^ 2 

Low  12.8%  8.4%  18.7% 

Medium  12.1% 

High  12.1%  16.5%  6.7% 

Overall  Average  .Suboptimality  =  12.5% 
Table  X:  Average  suboptimality  of  the  corridor  policy:  Setup  time  problem. 


Backorder  Cost     Setup  Time     Traffic  Intensity 
6 s () 


Low  9.5%  '         9.8%  14.0% 

Medium  9.8% 

High  11.8%  11.6%  8.2% 

Overall  Average  Suboptimality  =  10.7% 
Table  XI:  Average  suboptimality  of  the  generalized  base  stock  policy:  Setup  time  problem. 

will  increase  while  producing  product  2;  that  is,  the  increase  in  product  2's  inventory 
workload  will  exceed  the  reduction  in  product  Ts  inventory  workload.  The  optimal  policy 
takes  advantage  of  this  imbalance  by  allowing  product  2's  inventory  to  grow  beyond  the 
product  1  idling  threshold;  this  extra  product  2  inventory  allows  the  server  to  idle  while 
setup  for  product  1. 

Performance  of  the  corridor  policy.  The  corridor  policy  exhibits  erratic  behavior. 
The  policy  performs  very  well  in  the  symmetric  setup  cost  cases  (it  outperforms  the 
proposed  policy  in  six  of  the  18  scenarios,  all  of  which  have  low  or  medium  utilizations), 
but  degrades  slightly  at  high  utilization.  A  comparison  of  Figures  2  and  3  leads  us  to 
believe  that  the  parameters  of  the  policy  are  being  set  correctly  at  high  utilizations,  and 
the  performance  degradation  is  due  to  the  corridor  policy's  inability  to  employ  mean  lot 
sizes  that  are  state-dependent. 

The  corridor  policy  does  not  perform  as  well  in  the  symmetric  setup  time  cases;  it 
is  not  able  to  increase  the  cycle  length  r  for  small  total  workloads  and  so  has  difficulty 
recovering  from  this  high  backorder  region.  In  contrast  to  the  symmetric  setup  cost  cases, 
the  corridor  policy's  performance  diminishes  in  light  traffic;  we  have  not  determined  how 
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Figure  5:  Switching  curves  for  a  symmetric  setup  time  case. 

much  of  this  degradation  is  due  to  the  inaccuracy  of  the  heavy  traffic  approximation  at 
low  utilizations,  and  how  much  is  intrinsic  to  the  policy. 

The  corridor  policy  performs  nuich  worse  when  asymmetry  is  present:  Its  suboptim- 
ality  is  13.1%  and  14.9%  in  the  setup  cost  cases  and  increases  to  30.6%  and  60.9%i  in 
the  setup  time  cases.  Comparing  Figures  2,  4  and  6,  it  would  appear  that  the  corridor 
policy  would  never  be  very  close  to  optimal  for  an  asymmetric  problem.  In  fact,  Figure 
6  suggests  that  a  hyperplane  corridor  policy  (see  Figure  7  of  Sharifnia,  Caramanis  and 
Gershwin)  might  perform  reasonably  well  in  the  asymmetric  setup  time  problem;  in  the 
two-product  case,  the  two  lines  forming  the  corridor  in  Figure  2  would  not  be  parallel 
in  the  hyperplane  corridor  policy,  but  would  intersect  at  an  idling  point  in  the  upper 
right  portion  of  the  graph  and  generate  a  cone-shaped  corridor  emanating  out  in  the 
southwesterly  direction. 

Performance  of  the  generalized  base  stock  policy.  The  generalized  base  stock  policy 
performs  better  in  the  setup  time  cases  than  in  the  setup  cost  cases:  Its  average  suboptim- 
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Figure  6:  Switching  curves  for  the  6i  =  10  asymmetric  setup  time  case. 


ality  is  23.6%  for  the  18  symmetric  setup  cost  scenarios  and  10.7%  for  the  12  symmetric 
setup  time  cases.  In  contrast  to  the  corridor  policy,  the  generalized  base  stock  policy's 
use  of  large  lot  sizes  when  the  total  workload  is  negative  is  a  key  reason  for  its  ability  to 
avoid  poor  performance  in  the  symmetric  setup  time  cases;  however,  these  large  lot  sizes 
lead  to  considerable  backordering  in  the  setup  cost  scenarios.  Like  the  corridor  policy,  the 
generalized  base  stock  policy's  performance  deteriorates  at  high  utilizations  in  the  setup 
cost  cases  and  at  low  utilizations  in  the  setup  time  cases. 

The  generalized  base  stock  policy's  suboptimality  is  .35.8%  and  45.8%  in  the  asym- 
metric setup  cost  cases  and  11.5%  and  15.8%i  in  the  asymmetric  setup  time  case.  It  is 
interesting  to  note  that  the  generalized  base  stock  policy  handles  expensive  inventory  in  a 
manner  opposite  to  that  of  the  proposed  policy.  The  order-up-to  level  of  the  most  costly 
good  is  set  larger  than  those  of  less  expensive  products  to  reduce  the  risk  of  expensive 
backordering;  in  contrast,  the  proposed  policy  minimizes  the  excess  or  deficit  amounts 
of  expensive  inventory.   It  is  clear  that  the  generalized  base  stock  policy  is  incapable  of 


closely  approximating  the  optimal  setup  cost  solution  in  Figure  4. 

Five-product  examples:  In  the  six  setup  cost  cases  in  Table  VI,  the  proposed  and 
corridor  policies  are  roughly  comparable  in  the  two  A'  =  500  symmetric  cases  and  the 
corridor  policy  is  about  10%  more  costly  in  the  two  A'  =  50  symmetric  cases  and  about 
8%  more  expensive  in  the  two  asymmetric  cases.  The  generalized  base  stock  policy  does 
not  fare  as  well  in  the  four  symmetric  setup  cost  cases,  incurring  an  18.7%  cost  increase 
relative  to  the  proposed  policy,  on  average.  Once  again,  the  generalized  base  stock  policy 
performs  very  po'^-ly  in  the  asymmetric  setup  cost  cases. 

In  contrast,  the  generalized  base  stock  policy  performs  slightly  better  than  the  pro- 
posed policy  in  the  two  symmetric  setup  time  cases  in  Table  VI,  and  is  about  8%  more 
costly  than  the  proposed  policy  in  the  two  asymmetric  cases.  Both  of  these  policies  out- 
perform the  corridor  policy  in  the  four  setup  time  cases.  The  corridor  policy  is  about 
12%)  more  costly  than  the  proposed  policy  in  the  two  asymmetric  cases,  but  performs 
extremely  poorly  in  one  of  the  two  symmetric  cases. 

To  compare  the  relative  cost  differences  in  the  two-product  cases  and  the  five-product 
cases,  we  can  identify  the  six  symmetric  cases  in  Table  VI  with  their  two-product  coun- 
terparts in  Tables  II  and  VIII;  for  example,  the  first  scenario  in  Table  VI  corresponds 
to  the  6  =  5.  A'  =  20.  p  =  0.9  case  in  Table  II.  For  the  four  setup  cost  cases,  the  cost 
increases  of  the  straw  policies  relative  to  the  proposed  policy  are  somewhat  larger  for 
the  two-product  cases:  the  generalized  base  stock  policy's  average  cost  increase  is  5.8% 
for  the  two-product  cases  versus  4.7%i  for  the  five-product  cases,  and  the  corresponding 
quantities  for  the  generalized  base  stock  policy  are  26.3%  and  18.7%,  respectively.  For  the 
two  setup  time  cases,  the  average  cost  increase  of  the  generalized  base  stock  policy  is  2.7% 
for  the  two-product  scenarios  and  -l.l%i  for  the  five-product  cases.  Disregarding  the  poor 
performance  of  the  corridor  policy  in  one  of  the  five-product  symmetric  setup  cost  scen- 
arios, it  appears  that  the  relative  cost  advantage  of  the  proposed  policy  degrades  slightly 
when  the  number  of  products  increases  from  two  to  five;  however,  further  experiments 
are  required  to  fully  investigate  this  issue. 

LacA'  of  Robustness.    Simulation  results  not  reported  here  show  that  the  performance 
of  the  three  policies  are  rather  sensitive  to  the  policy  parameters,  particularly  in  the  setup 
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time  problem;  this  is  somewhat  surprising,  given  the  robustness  of  some  simpler  models 
(e.g.,  the  EOQ  model)  that  capture  the  tradeoff  between  inventory  costs  and  setups. 
Because  it  is  unable  to  increase  its  lot  sizes  as  the  total  inventory  decreases,  the  corridor 
policy  is  clearly  the  least  robust  of  the  three  policies:  if  the  corridor  width  is  set  too 
narrow  (as  apparently  happened  in  the  eighth  row  of  Table  VI)  then  stability  problems 
can  set  in  (notice  the  confidence  intervals  for  this  case). 

The  s/y/n  Refinement.  Recall  that  the  s/ s/n  refinement  described  in  §2.7  is  incorpor- 
ated into  the  proposed  policy,  but  not  the  two  straw  policies.  We  tested  all  three  policies 
with  and  without  the  refinement,  and  summarize  our  findings  here.  The  refinement  had 
a  minor  effect  on  the  performance  of  the  proposed  policy  in  the  p  =  0.9  cases;  however, 
by  decreasing  the  cycle  length,  it  significantly  improved  performance  in  the  lower  utiliz- 
ation cases.  The  refinement  had  a  mixed  influence  on  the  generalized  base  stock  policy, 
sometimes  improving  and  sometimes  degrading  performance;  overall,  it  slightly  impaired 
performance.  The  refinement  had  a  negative  effect  on  the  corridor  policy,  and  let  to  a 
severe  stability  problem  in  the  eighth  row  of  Table  VI. 

Summary.  Although  additional  asymmetric  cases  need  to  be  investigated  before 
drawing  definitive  conclusions  for  the  two-product  problems,  our  observations  can  be 
summarized  as  follows. 

The  proposed  policy  performs  very  well  in  the  34  two-product  cases:  Figures  3  to  6 
confirm  that  it  captures  nearly  all  of  the  complexities  of  the  optimal  policy,  its  suboptim- 
ality  is  6.0%  over  the  34  cases  (and  2.1%  over  the  12  cases  that  do  not  obviously  violate 
the  heavy  traffic  conditions),  and  it  is  quite  robust  with  respect  to  the  heavy  traffic  condi- 
tions, especially  considering  that  most  potential  industrial  applications  for  the  SELSP  are 
in  settings  with  high  traffic  intensities.  However,  the  relative  superiority  of  the  proposed 
policy  appears  to  degrade  slightly  as  the  number  of  products  increases,  and  this  issue 
requires  further  investigation. 

The  two  straw  policies  are  not  flexible  enough  to  consistently  capture  the  subtleties  of 
the  optimal  policy.  The  corridor  policy  outperforms  the  generalized  base  stock  policy  in 
24  of  the  34  two-product  examples,  and  its  average  suboptimality  is  11.2%i  as  compared 
to  19.5%)  for  the  generalized  base  stock  policy.   Nonetheless,  in  the  setup  time  cases  the 


corridor  policy  fails  to  use  large  lot  sizes  when  the  total  workload  is  negative  and  can 
perform  erratically  (see  Table  VIII).  The  generalized  base  stock  policy  is  never  close  to 
optimal  and  performs  poorly  in  the  asymmetric  setup  cost  cases. 

4.  CONCLUDING  REMARKS 

From  a  practical  standpoint,  the  stochastic  economic  lot  scheduling  problem  is  argu- 
ably the  most  important  scheduling  problem  in  operations  management.  Unfortunately, 
there  has  been  no  success  in  obtaining  an  optimal  solution  to  this  problem  using  standard 
techniques  (e.g.,  semi-Markov  decision  process  theory).  In  this  paper  we  restrict  ourselves 
to  the  class  of  dynamic  cyclic  policies,  where  the  server  has  three  options  at  each  point 
in  time:  Idle,  produce  the  product  that  is  currently  set  up,  or  switch  over  to  the  next 
product  in  the  fixed  sequence.  Using  heavy  traffic  analysis,  in  particular  the  averaging 
principles  derived  by  Coffman,  Puhalskii  and  Reiman,  we  make  considerable  progress  on 
this  problem:  For  the  setup  cost  problem,  the  optimal  heavy  traffic  lot-sizing  policy  is 
derived  in  closed  form,  and  the  idleness  policy  is  reduced  to  the  numerical  calculation  of 
a  single  threshold  value.  For  the  setup  time  problem,  some  key  qualitative  characteristics 
of  the  optimal  heavy  traffic  policy  are  derived;  moreover,  regardless  of  the  number  of 
products,  we  reduce  the  problem  to  a  one-dimensional  diffusion  control  problem  that  is 
solved  numerically. 

The  explicitness  of  our  results,  coupled  with  the  surprisingly  intricate  behavior  of 
the  optimal  policy,  leads  to  a  number  of  new  insights  into  the  optimal  solution  to  the 
SELSP.  These  insights  are  summarized  in  §1.8  and  §2.5,  and  describe  how  the  dynamic 
lot-sizing  policy  depends  upon  whether  (i)  setup  costs  or  setup  times  are  incurred,  (ii)  the 
cost  structure  is  symmetric  or  asymmetric  across  products  and  (iii)  the  total  workload 
embodied  in  the  current  finished  goods  inventory  is  much  less  than  zero,  in  a  neighborhood 
containing  zero,  or  larger  than  zero  and  near  the  optimal  idling  threshold. 

We  also  perform  a  heavy  traffic  analysis  of  two  classes  of  policies  that  are  closely 
related  to  ones  analyzed  by  Federgruen  and  Katalan  (1993)  and  Sharifnia,  Caramanis 
and  Gershwin:  the  unified  treatment  of  the  optimal  policy  and  the  two  straw  policies 
(see.  in  particular,  Figures  2  through  6)  makes  transparent  the  relative  strengths  and 
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weaknesses  of  the  straw  policies.  A  computational  study  is  undertaken  to  compare  the 
proposed  policy  and  these  two  straw  policies  to  the  numerically  computed  optimal  policy 
in  34  two-product  examples.  The  computational  study  (see  §3.4  for  a  description  of  the 
key  observations)  confirms  that  the  insights  summarized  in  §1.8  and  §2.5  do  indeed  occur 
in  the  optimal  policy.  Moreover,  numerical  results  for  the  two-product  examples  show  that 
the  proposed  policy  is  reasonably  close  to  optimal,  is  robust  with  respect  to  the  heavy 
traffic  conditions,  and  outperforms  the  two  straw  policies;  in  contrast,  the  two  straw 
policies  lack  the  sophistication  required  to  imitate  the  subtleties  of  the  optimal  policy,  and 
their  behavior  is  somewhat  erratic. 

The  computational  study  also  includes  10  five-product  examples,  and  the  results  sug- 
gest that  the  relative  superiority  of  the  proposed  policy  degrades  slightly  as  the  number 
of  products  increases.  In  the  last  decade,  researchers  have  improved  the  accuracy  and 
robustness  of  heavy  traflfic  approximations  through  the  use  of  various  heuristic  refine- 
ments (e.g.,  Harrison  1988,  Nguyen  1995,  Lennon  1995).  Although  we  have  investigated 
one  such  refinement  (see  §2.7),  we  believe  that  other  refinements  to  the  proposed  policy 
may  lead  to  significant  improvements  and  should  be  investigated;  such  refinements  may 
be  necessary  in  order  to  develop  effective  policies  for  problems  with  many  products  and 
moderate  traffic  intensities. 

This  study  has  only  considered  dynamic  cyclic  policies,  where  each  class  is  served 
once  per  cycle  in  a  fixed  sequence.  More  generally,  the  cost  of  dynamic  periodic  policies 
(or  polling  tables),  where  each  product  can  be  produced  more  than  once  in  a  cycle,  can 
be  evaluated  using  the  methods  developed  here.  However,  we  have  not  yet  found  an 
efficient  method  for  optimizing  within  this  class  of  policies,  and  a  thorough  investigation 
of  dynamic  periodic  and  nonperiodic  policies  is  left  for  future  research. 
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